Latent Information Sharing for Accelerating Federated Learning

arXiv:2610.01126v1 Announce Type: new Abstract: Federated learning (FL) is a communication-efficient distributed learning paradigm. However, client drift remains one of the most critical challenges, hindering the efficient training of a global model. In this study, we propose a novel latent information sharing scheme that directly mitigates data heterogeneity across clients. Our theoretical and empirical results show that sharing a small amount of hidden-layer activations significantly improves training efficiency while preserving convergence guarantees and data privacy. Furthermore, we compare our method with existing FL approaches designed to address client drift, including FedProx, SCAFFOLD, FedPVR, FedProto, and SplitFed, and demonstrate superior model accuracy under a fixed round budget without incurring excessive communication overhead. Overall, this work presents a promising new knowledge aggregation scheme and provides a comprehensive analysis of the impact of activation sharing on federated optimization.

Related

• An Efficient Subspace Algorithm for Federated Learning on Heterogeneous Data
• FedFFT: Taming Client Drift in Federated SAM via Spectral Perturbation Filtering
• Subspace Optimization for Efficient Federated Learning under Heterogeneous Data

Source: arXiv cs.LG | 2026-10-02