InfoLaw: Information Scaling Laws for Large Language Models with Quality-Weighted Mixture Data and Repetition
DGX agentarXiv:2605.02364v1 Announce Type: new Abstract: Upweighting high-quality data in LLM pretraining often improves performance, but in datalimited regimes, especially under overtraining, stronger upweigh