Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals
arXiv:2610.11247v1 Announce Type: cross Abstract: On-policy distillation (OPD) enables effective capability transfer between language models, yet the mechanisms underlying its failures are not fully u