1% of Tokens Can Be Enough: On Gradient Estimation in On-Policy Distillation
DGX agentarXiv:2609.24432v1 Announce Type: cross Abstract: Sparse on-policy distillation (OPD) allocates teacher supervision to a small subset of tokens in student-generated trajectories. However, useful teach