Revisiting the Capacity Gap in Chain-of-Thought Distillation from a Practical Perspective

arXiv:2604.08880v1 Announce Type: cross Abstract: Chain-of-thought (CoT) distillation transfers reasoning behaviors from a strong teacher to a smaller student, but prior work reports a capacity gap: distillation may fail when the teacher-student capability mismatch is large. We revisit the capacity gap from a practical perspective by re-examining commonly used experimental settings. Notably, we find that CoT distillation often degrades performance compared to the student's pre-distillation baseline, an issue obscured when only post-distillation comparisons are reported. We therefore propose a more realistic evaluation protocol and find that the impact of capacity gap effects does not consistently dominate across tasks and settings, especially when candidate teachers differ substantially in performance. Our results offer practical guidance for selecting teacher-student pairs in CoT distillation.

Related

• On the Step Length Confounding in LLM Reasoning Data Selection
• The Detection-Extraction Gap: Models Know the Answer Before They Can Say It
• SHAPE: Stage-aware Hierarchical Advantage via Potential Estimation for LLM Reasoning
• Reasoning Fails Where Step Flow Breaks

Source: arXiv cs.AI | 2026-04-13