Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging
DGX agentarXiv:2602.03702v2 Announce Type: replace-cross Abstract: Large language models are increasingly trained in continual or open-ended settings, where the total training horizon is not known in advance.