Tutorials
Learning to Coach for Experiential Learning
arXiv:2609.15851v1 Announce Type: new Abstract: Language models can learn from experience, but raw solution trajectories are often too long and noisy to provide effective guidance. In this work, we pr
arXiv:2609.15851v1 Announce Type: new Abstract: Language models can learn from experience, but raw solution trajectories are often too long and noisy to provide effective guidance. In this work, we propose Learning to Coach (L2C), a framework that trains a dedicated LLM-as-a-Coach to extract actionable experiential knowledge from an actor model's previous trajectory. The actor remains frozen, while the LLM-as-a-Coach is trained to maximize a reward given by the correctness of the actor's guided response. We study two such rewards: a same-instance reward, which improves subsequent responses on the original problem, and a cross-instance reward, which elicits knowledge that transfers to other instances. Across mathematical reasoning and interactive text-games, L2C consistently outperforms self-refinement and an untrained LLM-as-a-Coach. Running experiential learning for more iterations further improves accuracy and uses additional inference compute more effectively than enlarging the actor's decoding budget. The trained LLM-as-a-Coach also transfers to out-of-distribution tasks and adapts its guidance to the specific actor it coaches.
Related
- Learning What to Learn: Stage-Specific Data Sets for SFT-then-RL in Small Language Model Reasoning
- Towards Just-in-Time Adaptive Feedback: Enhancing Student Learning via Knowledge-Grounded LLM
- MentorCollab: Selective Large-to-Small Inference-Time Guidance for Efficient Reasoning
Source: arXiv cs.CL | 2026-09-15