Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
DGX agentarXiv:2609.11917v1 Announce Type: cross Abstract: As the supply of human-written text is exhausted, it has become standard practice to repeat language model training data. Prior work has studied data