Time-Frequency Analysis for Neural Networks

arXiv:2512.15992v2 Announce Type: replace-cross Abstract: We develop a quantitative approximation theory for shallow neural networks using tools from time-frequency analysis. Working in weighted modulation spaces M^{p,q}m(mathbf{R}^{d}), we prove dimension-independent approximation rates in Sobolev norms W^{n,r}(Omega) for networks whose units combine standard activations with localized time-frequency windows. Our main result shows that for f in M^{p,q}m(mathbf{R}^{d}) one can achieve [ |f - f_N|{W^{n,r}(Omega)} lesssim N^{-1/2},|f|{M^{p,q}_m(mathbf{R}^{d})}, ] on bounded domains, with explicit control of all constants. We further obtain global approximation theorems on mathbf{R}^{d} using weighted modulation dictionaries, and derive consequences for Feichtinger's algebra, Fourier-Lebesgue spaces, and Barron spaces. Numerical experiments in one and two dimensions confirm that modulation-based networks achieve substantially better Sobolev approximation than standard ReLU networks, consistent with the theoretical estimates.

Related

• Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix
• On the Convergence Analysis of Muon
• Universal Approximation with XL MIMO Systems: OTA Classification via Trainable Analog Combining

Source: arXiv cs.LG | 2026-04-14