One LR Doesn't Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs
DGX agentarXiv:2605.22297v1 Announce Type: new Abstract: Learning rate configuration is a fundamental aspect of modern deep learning. The prevailing practice of applying a uniform learning rate across all laye