Generalization and Scaling Laws for Mixture-of-Experts Transformers
arXiv:2604.09175v1 Announce Type: cross Abstract: We develop a theory of generalization and scaling for Mixture-of-Experts (MoE) Transformers that cleanly separates active per-input capacity from rout