Hyperparameter Scaling Laws Across MoE Sparsity
DGX agentarXiv:2609.08690v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models expand model capacity without a proportional increase in training compute, but increasing sparsity makes reliable hyperp