Knowledge Distillation under Teacher Misspecification: An Order-Parameter Analysis of the Gap between Teacher Mimicry and Task Performance
DGX agentarXiv:2608.29472v1 Announce Type: cross Abstract: Knowledge distillation trains a small student model to reproduce the outputs of a large teacher model, and its progress is typically monitored through