Research
Loop Corrections to the Training and Generalization Errors of Random Feature Models
arXiv:2604.12827v1 Announce Type: cross Abstract: We investigate random feature models in which neural networks sampled from a prescribed initialization ensemble are frozen and used as random features
arXiv:2604.12827v1 Announce Type: cross Abstract: We investigate random feature models in which neural networks sampled from a prescribed initialization ensemble are frozen and used as random features, with only the readout weights optimized. Adopting a statistical-physics viewpoint, we study the training, test, and generalization errors beyond the mean-kernel approximation. Since the predictor is a nonlinear functional of the induced random kernel, the ensemble-averaged errors depend not only on the mean kernel but also on higher-order fluctuation statistics. Within an effective field-theoretic framework, these finite-width contributions naturally appear as loop corrections. We derive the loop corrections to the training, test, and generalization errors, obtain their scaling laws, and support the theory with experimental verification.
Related
- Continuous-time Online Learning via Mean-Field Neural Networks: Regret Analysis in Diffusion Environments
- Wolkowicz-Styan Upper Bound on the Hessian Eigenspectrum for Cross-Entropy Loss in Nonlinear Smooth Neural Networks
- Training Deep Visual Networks Beyond Loss and Accuracy Through a Dynamical Systems Approach
- Bi-Level Optimization for Single Domain Generalization
Source: arXiv cs.AI | 2026-04-15