Safety
DualOPSD: Adaptive Privileged Teachers for On-Policy Self-Distillation
arXiv:2608.26019v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) uses a privileged copy of the student model to provide dense supervision without an external teacher. OPSD keeps this
arXiv:2608.26019v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) uses a privileged copy of the student model to provide dense supervision without an external teacher. OPSD keeps this privileged teacher fixed, even though the student distribution and output style change during training. We propose DualOPSD, an asymmetric alternating framework that adapts both policies. The student first learns from the privileged teacher. The teacher then moves toward the updated student distribution on the same student trajectory. This update makes later supervision responsive to the learner and does not require another rollout. On Qwen3-8B in non-thinking mode, DualOPSD improves avg@12 over OPSD by 23.61, 13.89, and 10.00 points on AIME 2024, AIME 2025, and HMMT 2025. Results at 1.7B and 4B show that the accuracy gain depends on model scale. Across all three scales, DualOPSD reduces truncation. The 4B diagnostic also shows lower KL in both directions between the teacher and student.
Related
- Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoning
- Rethinking Privileged Information in On-Policy Self-Distillation
- PAST: Privileged Adaptation from Complete Student Trajectories for On-Policy Self-Distillation
- When Privileged Guidance Misaligns: State-Matched Routing and Contextualized Self-Distillation for Multi-Turn Agents
Source: arXiv cs.LG | 2026-08-27