Trust the Direction, Search the Step: Zero-and-First-Order Methods for LLM Fine-Tuning
DGX agentarXiv:2610.02190v1 Announce Type: new Abstract: Step-size selection remains a central challenge in large-scale neural network optimization; conservative steps slow convergence, while aggressive steps