Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization
DGX agentarXiv:2605.04269v1 Announce Type: cross Abstract: We provide a theoretical analysis of Adam under non-stationary stochastic objectives, separating two regimes: Euclidean tracking under adaptive strong