Model Releases
The Geometry of Statistical Feature Learning in Mean-Field Langevin Dynamics
arXiv:2606.31429v2 Announce Type: cross Abstract: We introduce a geometric formulation of statistical feature learning for supervised regression. Feature learning is defined through a base--fiber deco
arXiv:2606.31429v2 Announce Type: cross Abstract: We introduce a geometric formulation of statistical feature learning for supervised regression. Feature learning is defined through a base--fiber decomposition: the base is the feature-side geometry produced by training, and the fiber is the learned feature space where estimation is performed. We prove this property for spherical mean-field Langevin dynamics, viewed as the Wasserstein gradient flow of a negative entropy-regularized empirical risk. In Gaussian multi-index models, the low-temperature stationary distribution concentrates near the hidden indices, forms a multi-spike structure, and yields parameter recovery with high probability, even though negative entropy regularization penalizes concentration. This concentration has a sharp transition at temperature lambdaasymp 1. In Gaussian single-index models, the stationary measure satisfies a concentration property, with parity determining whether it lives on S_2^{d-1} or RP^{d-1}. The induced learned feature space aligns the regression signal and yields rates d/N and Md/N, up to logarithmic factors.
Related
- Feature Learning in Wide Neural Networks under muP: Identifiability and Sparse-Dictionary Decomposition of the Mean-Field Limit
- Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks
- Fisher Widths: Local Learning Geometry and Anisotropic Recovery
Source: arXiv cs.LG | 2026-10-07