Removing Sandbagging in LLMs by Training with Weak Supervision

arXiv:2604.22082v1 Announce Type: cross Abstract: As AI systems begin to automate complex tasks, supervision increasingly relies on weaker models or limited human oversight that cannot fully verify output quality. A model more capable than its supervisors could exploit this gap through sandbagging, producing work that appears acceptable but falls short of its true abilities. Can training elicit a model's best work even without reliable verification? We study this using model organisms trained to sandbag, testing elicitation techniques on problem-solving math, graduate-level science, and competitive coding tasks. We find that training with weak supervision can reliably elicit sandbagging models when supervised fine-tuning (SFT) and reinforcement learning (RL) are combined: SFT on weak demonstrations breaks the sandbagging behavior, enabling RL to then fully elicit performance. Neither method succeeds reliably alone-RL without SFT almost always leads to reward hacking rather than genuine improvement. Critically, this relies on training being indistinguishable from deployment; when models can distinguish between training and deployment, they can perform well during training while continuing to sandbag afterward. Our results provide initial evidence that training is a viable mitigation against sandbagging, while highlighting the importance of making training indistinguishable from deployment.

Related

• CoAuthorAI: A Human in the Loop System For Scientific Book Writing
• Evaluating LLMs as Human Surrogates in Controlled Experiments
• Detecting and refurbishing ground truth errors during training of deep learning-based echocardiography segmentation models
• Harmonizing Multi-Objective LLM Unlearning via Unified Domain Representation and Bidirectional Logit Distillation
• Beyond Surface Statistics: Robust Conformal Prediction for LLMs via Internal Representations

Source: arXiv cs.AI | 2026-04-27