Moment-Accurate Gaussian Mixtures for Constant-Step Stochastic Approximation

arXiv:2610.05595v1 Announce Type: cross Abstract: Local Gaussian models of constant-step learning predict output variability and expected losses, but weak convergence alone does not justify these moment predictions. We establish moment-accurate Gaussian mixtures by matching stationary energy with local Ornstein--Uhlenbeck limits, ruling out quadratic tail mass invisible to weak convergence. For step size a, the second-order Wasserstein error is o(sqrt a), uniformly over invariant laws, using each law's actual root weights. The assumptions combine confinement, descent, finitely many hyperbolic equilibria and root continuity with finite-variance innovations. The result yields observable covariances, expected objective gaps and first-order mean shifts, while allowing singular covariances, compatible saddles and weights without a limit. For additive noise given by a fixed invertible transform of independent standardized Student t_3 coordinates, symmetry gives an order-sharp sqrt a smooth-test bound. Numerical transport calculations demonstrate the value of root-specific covariances; controlled SGD studies assess observable predictions across step sizes, batch sizes and model geometries.

Related

• Gaussian Approximation and Multiplier Bootstrap for Federated Linear Stochastic Approximation
• Gaussian Limits for SGD Without Stationary Moments
• SAUSS: Stochastic Approximation with Unbiased Simulated Scores for Limited Dependent Variable Models

Source: arXiv cs.LG | 2026-10-06