Why Does Adaptive Batching Help LLM Pretraining? A Perspective from Unbounded Variance
arXiv:2610.02355v1 Announce Type: new Abstract: Increasing the batch size during training is a common practice in large language model (LLM) pretraining, yet the theoretical justification behind its s