SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
DGX agentarXiv:2605.08738v1 Announce Type: cross Abstract: Structured pruning and knowledge distillation (KD) are typical techniques for compressing large language models, but it remains unclear how they shoul