Residual neural networks overcome the curse of dimensionality for semilinear heat equations

Summary

This paper proves the first expressivity result showing that residual neural networks (ResNets) overcome the curse of dimensionality in the numerical approximation of semilinear heat equations ∂u/∂t = ½Δu + f(u) with globally Lipschitz, gradient-independent nonlinearities. Theorem 5.1 establishes that, under polynomial growth and network-approximability hypotheses on the PDE data, there exist ResNets Ψ_{d,ε} with at most η·d^η·ε^{-η} parameters whose realizations approximate the dimension-d solution in L² on [0,1]^d within error ε. The proof represents one deterministic realization of a multilevel Picard (MLP) estimator as a ResNet: shortcut connections transmit the spatial variable and a scalar accumulator, while residual branches successively add the estimator's summands (Lemma 4.8, Proposition 4.9). Crucially, the construction requires no activation that represents the identity, admitting bounded monotone sigmoidal activations — a genuine generalization over the feedforward precursors [36], [1]. For ridge-sum initial conditions and globally Lipschitz truncations of f, an explicit bound C_ξ·d^{4+ξ}·ε^{-(3+ξ)} holds for every ξ>0, illustrated by an Allen–Cahn-type example. Stated limitations: the result is existence-only (no training guarantee), ResNet depth grows with d and ε^{-1}, the untruncated Allen–Cahn cubic is excluded, and the explicit exponent covers only ridge-sum data; gradient-dependent nonlinearities and space-time error are left open.