The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton
arXiv:2510.09378v2 Announce Type: replace Abstract: Recent efforts to accelerate LLM pretraining have focused on computationally-efficient approximations that exploit second-order structure. This rais