Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples

arXiv:2608.06687v1 Announce Type: cross Abstract: We develop a rigorous theory of discrete residual least-squares approximation for elliptic spectral equations mathfrak L_eta u=f using linearized ReLU^k neural networks on the sphere, where mathfrak L_eta is a positive elliptic spectral multiplier of order eta. Given a parameter set Theta_n={heta_{j}^}_{j=1}^nsubsetmathbb S^d, we approximate u in the linearized network space L_n^k(Theta_n) by the discrete residual on the collocation points {eta_i^}{i=1}^m egin{equation*} u{n,m}inargmin_{v_nin L_n^k(Theta_n)}frac1msum_{i=1}^mleft(f(eta_i^)-mathfrak L_eta v_n(eta_i^)right)^2. end{equation*} With k>frac{d-1}{2}+eta, for antipodally quasi-uniform network parameter sets and any quasi-uniform collocation points with mgtrsim n, we prove that egin{equation*} |u-u_{n,m}|{mathcal H^{eta}(mathbb S^d)}eqsim|f-mathfrak L_eta u{n,m}|{mathcal L^2(mathbb S^d)}lesssim n^{-frac{r}{d}} egin{cases} |f|{mathcal W^{r,p}(mathbb S^d)},&frac{d}{p}2, |f|{mathcal H^r(mathbb S^d)},&r>frac{d}{2}. end{cases} end{equation*} We also establish a high-probability residual estimate, up to a logarithmic factor and an arbitrarily small smoothness loss, for i.i.d. uniformly distributed collocation points. The key analytical ingredient is a Bernstein inequality for linearized ReLU^k network spaces. If nderline h denotes the antipodal separation distance of the network parameters, then egin{equation*} |v_n|{mathcal H^r(mathbb S^d)}lesssimnderline h^{-(r-s)}|v_n|_{mathcal H^s(mathbb S^d)},qquad 0leq s<r<k+frac12. end{equation*}

Source: arXiv cs.LG | 2026-08-10