SparseOpt: Addressing Normalization-induced Gradient Skew in Sparse Training
DGX agentarXiv:2605.27541v1 Announce Type: new Abstract: Dynamic Sparse Training (DST) methods train neural networks by maintaining sparsity while dynamically adapting the network topology. Despite the promise