Predictable Scaling Laws of Optimal Hyperparameters for LLM Continued Pre-training
DGX agentarXiv:2606.05610v1 Announce Type: new Abstract: The efficacy of continued pre-training for Large Language Models (LLMs) hinges upon hyperparameter configurations, such as learning rate and batch size.