Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devices

arXiv:2610.05915v1 Announce Type: new Abstract: This paper revisits the distributed learning problem for training a multinomial logistic regression model with the Federated Averaging (exttt{FedAvg}) algorithm. We concentrate on a scenario with arbitrarily large stepsizes and heterogeneous update rules where the devices may perform a different number of local updates in each round. We show that, with linearly separable data, exttt{FedAvg} is stable with any stepsizes and the objective values converge to zero at the rate of {al O}(1/R), where R is the number of communication rounds. Our result also demonstrates that the effects of device heterogeneity vanish asymptotically. For sufficiently large R, the objective values decrease monotonically and is bounded by {al O}( 1 / (R T_{rm avg})), where T_{rm avg} is the average number of local update steps per communication round across devices. Numerical experiments support our findings.

Related

• Improved Convergence of Large Stepsize Gradient Descent for Logistic Regression
• Tight Transition Time Bounds for Separable Logistic Regression at the Edge of Stability
• SGD in Multiclass Logistic Regression: Sequential Learning and Scaling Laws

Source: arXiv cs.LG | 2026-10-06