Bregman meets Levy: Stochastic mirror descent with heavy-tailed noise in continuous and discrete time

arXiv:2606.03769v1 Announce Type: cross Abstract: We study the robustness of stochastic mirror descent (SMD) under heavy-tailed noise, focusing on whether the method retains its convergence guarantees when run with infinite-variance stochastic gradient input. To address this question in a principled manner, we begin by introducing a continuous-time model of SMD as a stochastic differential equation (SDE) driven by a centered Levy noise process with finite p-th order moments, 1 < p leq 2. This scheme -- which we call the Levy mirror flow (LMF) -- arises naturally as the scaling limit of SMD in the presence of heavy-tailed noise. In particular, when p < 2 -- the heavy noise regime -- the trajectories of LMF generically exhibit jump discontinuities of arbitrary magnitude which, if frequent enough, lead to infinite variance. Nonetheless, despite this highly singular behavior, we show that LMF attains epsilon-optimality within O(epsilon^{-p/(p-1)}) time in the convex case, and within ilde O(epsilon^{-1/(p-1)}) time for (relatively) strongly convex objectives. These guarantees provide a transparent characterization of the impact of frequent long jumps on the convergence of the process, and percolate to a series of matching discrete-time guarantees for several variants of SMD under heavy-tailed noise.

Source: arXiv cs.LG | 2026-06-03