Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts
arXiv:2605.12476v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (SMoE) models enable scaling language models efficiently, but training them remains challenging, as routing can collapse ont