Research
Sharp Stationary Gaussian Approximation for Constant-Stepsize SGD
arXiv:2609.39144v1 Announce Type: new Abstract: We prove a sharp Gaussian approximation for the invariant law of constant-stepsize SGD with bounded additive noise generated by an exogenous uniformly e
arXiv:2609.39144v1 Announce Type: new Abstract: We prove a sharp Gaussian approximation for the invariant law of constant-stepsize SGD with bounded additive noise generated by an exogenous uniformly ergodic Markov chain. For a smooth, strongly convex objective with a Lipschitz Hessian and nondegenerate long-run noise covariance, the centered iterate normalized by the square root of the stepsize is O(sqrt{alpha})-close in 1-Wasserstein distance to its limiting Gaussian. The proof combines blockwise Gaussian comparison with long-run contraction. A four-state example gives a matching lower bound although the one-time noise marginal is symmetric and every nonzero-lag autocovariance vanishes. In this example, an adjacent third-order mixed moment produces the leading correction.
Related
- Constant-Stepsize Stochastic Approximation: Finite-Time Convergence, Gaussian Approximation, and Tail Bounds
- How Accurately Can a Gaussian Approximate Stochastic Approximation Iterates?
- Gaussian Approximation and Multiplier Bootstrap for Stochastic Gradient Descent
Source: arXiv cs.LG | 2026-10-01