The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes
DGX agentarXiv:2605.11182v2 Announce Type: replace Abstract: On-policy distillation (OPD) and on-policy self-distillation (OPSD) have emerged as promising post-training methods for large language models, offer