ExpertMuon-Compass: Alignment-Guided Step Sizes for Mixture-of-Experts Training
DGX agentarXiv:2610.04140v1 Announce Type: new Abstract: Mixture-of-experts (MoE) language models send each token to a few experts, so each expert is trained on a different part of the data, and this part chan