Precise Convergence Speed of Clipped SGD

arXiv:2609.29458v1 Announce Type: new Abstract: We present a tightened convergence analysis of clipped gradient descent on (L_0, L_1)-smooth functions, with quantitative constants. Building on the ideas of Koloskova et al (2023), we refactor several case disjunctions to reveal the central role of a control of the bias derived from fundamental properties of ell_2-projection, simplifying proofs. We also extend the domain of validity from eta leq 1 / (9 eta) to eta < 1 /eta where eta = L_0 + c L_1 for clipping constant c, which matches the more traditional analysis of smooth functions. We strengthen the convergence criterion from left( min_{t < T} E[lVert nabla f(x_t) rVert_2] right) to left( frac{1}{T} sum_{t < T} E[lVert nabla f(x_t) rVert_2] right) with matching speed, and lower the final achievable loss from O(min(sigma^2/c, sigma)) to the more precise 6 min(sigma^2 /c, 3 sigma).

Source: arXiv cs.LG | 2026-09-25