On-Policy Distillation Teaches New Skills but Not New Knowledge
arXiv:2610.09639v1 Announce Type: new Abstract: On-policy distillation (OPD) strengthens language-model reasoning, yet whether students acquire new factual knowledge or compositional skill for multi-s