Scaling Repeated-Data Pretraining: Batch Size, Regularization, and GPU-Time Tradeoffs
DGX agentarXiv:2610.15965v1 Announce Type: new Abstract: We study pretraining when compute is abundant but unique training data are finite, requiring repeated training on the same data. We first revisit infini