Understanding Off- vs On-Policy Distillation: A Tale of Distinct Training Objectives
DGX agentarXiv:2609.38666v1 Announce Type: cross Abstract: On-policy distillation (OPD) learns from teacher feedback on student-generated responses and has shown promise in reducing forgetting relative to supe