Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference
DGX agentarXiv:2609.05275v1 Announce Type: new Abstract: Layer dropout (a.k.a. stochastic depth) has been shown to enable faster training, higher accuracy, and robustness to zero-shot layer pruning in both lan