MuLoCo: Muon is a practical inner optimizer for DiLoCo
DGX agentarXiv:2505.23725v3 Announce Type: replace Abstract: DiLoCo is a powerful framework for training large language models (LLMs), enabling larger optimal batch sizes and increased accelerator utilization