SAGE: Sign-Adaptive Gradient for Memory-Efficient LLM Optimization
DGX agentarXiv:2604.07663v2 Announce Type: replace Abstract: The AdamW optimizer, while standard for LLM pretraining, is a critical memory bottleneck, consuming optimizer states equivalent to twice the model's