STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules
DGX agentarXiv:2610.12599v1 Announce Type: new Abstract: Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures