Modeling AdaGrad, RMSProp, and Adam with Integro-Differential Equations
DGX agentarXiv:2411.09734v3 Announce Type: replace Abstract: In this paper, we propose a continuous-time formulation for the AdaGrad, RMSProp, and Adam optimization algorithms by modeling them as first-order i