Adaptivity via a Parallel Architecture for Stochastic Gradient Methods Adaptivity via a Parallel Architecture for Stochastic Gradient Methods Adaptivity via a Parallel Architecture for Stochastic Gradient Methods

arXiv:2607.28902v1 Announce Type: new Abstract: We develop a parallel framework that assembles static gradient methods to achieve better adaptivity. A static gradient method, denoted by GD(x_0,T), takes as input an initial point x_0inR^n and Tin R^+ specifying the number floor{T} of iterations. The step size is chosen as s=S(T), where S(dot) is a predetermined function of T. The method then performs the iterations x_{i+1}=x_i-frac{eta}{s}dot g_i, where g_i is a stochastic gradient evaluated at x_i, and eta is a scaling factor. For an integer pge1, the p processors in the proposed parallel framework search for an appropriate value of T according to a geometric sequence so that the resulting gradient descent satisfies the desired convergence conditions. Each processor executes an infinite sequence of stages indexed by i=1,2,ldots. At stage i, processor j is assigned T_{j,i}=h(j,i), where h:NimesN rightarrowR^{+} is a prescribed function. Processor j (j=0,1,ldots,p-1) executes GD(x_0, T_{j,i}) at stage i.

Source: arXiv cs.LG | 2026-08-03