Why Does Adaptive Batching Help LLM Pretraining? A Perspective from Unbounded Variance
DGX agentarXiv:2610.02355v1 Announce Type: new Abstract: Increasing the batch size during training is a common practice in large language model (LLM) pretraining, yet the theoretical justification behind its s