AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models

arXiv:2405.16240v3 Announce Type: replace Abstract: In this paper, we introduce analytic federated learning (AFL), a new training paradigm that brings analytical (i.e., closed-form) solutions to the federated learning (FL) with pre-trained models. Our AFL draws inspiration from analytic learning -- a gradient-free technique that trains neural networks with analytical solutions in one epoch. In the local client training stage, the AFL facilitates a one-epoch training, eliminating the necessity for multi-epoch updates. In the aggregation stage, we derive an absolute aggregation (AA) law. This AA law allows a single-round aggregation, reducing heavy communication overhead and achieving fast convergence by removing the need for multiple aggregation rounds. More importantly, the AFL exhibits a property that extit{invariance to data partitioning}, meaning that regardless of how the full dataset is distributed among clients, the aggregated result remains identical. This could spawn various potentials, such as data heterogeneity invariance and client-number invariance. We conduct experiments across various FL settings including extremely non-IID ones, and scenarios with a large number of clients (e.g., ge 1000). In all these settings, our AFL constantly performs competitively while existing FL techniques encounter various obstacles. Our codes are available at https://github.com/ZHUANGHP/Analytic-federated-learning.

Related

• Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach
• Extraction of linearized models from pre-trained networks via knowledge distillation
• Selective Neuron Amplification for Training-Free Task Enhancement
• DisCEdge: Distributed Context Management for Large Language Models at the Edge

Source: arXiv cs.LG | 2026-04-10