ReLU Neural Network Approximation to Smooth Functional Operator: Dimensional Decay and Error Analysis

arXiv:2609.15355v1 Announce Type: cross Abstract: We study the uniform approximation of smooth scalar-valued functionals on an infinite-dimensional separable Hilbert space by deep ReLU neural networks. Writing the functional input as X(t)=sum_{dgeq1}xi_dnu_d(t), we quantify the importance of coordinate d through w_ds_d, where s_d bounds the magnitude of the corresponding basis score and w_d controls the directional Frechet sensitivity of the target functional. Our constructive analysis combines coordinate truncation, anisotropic partitioning, local Taylor approximation, and ReLU network realization, while allowing unrestricted interactions among the retained coordinates. We establish a general nonasymptotic upper bound for the uniform approximation error and a complementary pseudo-dimension-based lower bound for the worst-case approximation error. Under generalized exponential coordinate decay w_ds_dasympexp(-cd^rho), with rho>0, the upper and lower bounds match at the leading order and thus yield the nearly optimal approximation rate, which is stretched-exponential in the logarithm of the network budget. This is the first work to characterize neural network approximation error for infinite-dimensional functional inputs explicitly through the joint dimensional decay of coordinate magnitudes and directional sensitivities.

Source: arXiv cs.LG | 2026-09-15