Safety
Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs
arXiv:2609.03955v1 Announce Type: new Abstract: Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback. The feedback for c
arXiv:2609.03955v1 Announce Type: new Abstract: Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback. The feedback for coding problems mainly comes from specific test cases, where high-quality test cases are often scarce since they should be both sound and discriminative. We thus turn to study the auto-generation of test cases using the learned model. We find this is naturally an adversarial RL problem: the model is expected to generate effective test cases as counterexamples, depending on the solver's current failure modes. We propose Test Cases Scaling (TCS), a two-stage RL framework for effective test generation. Both stages train a test generator from a rolling policy-aligned buffer: Stage 1 generates tests consistent with the reference solution, and Stage 2 restricts the buffer to current failure modes and learns counterexample tests. Across TACO and LiveCodeBench, TCS improves both pass@1 and inference-time answer selection according to generated tests. We find the learned test generator also enables effective selection among other LLM outputs.
Related
- LEAP: Lean Environment-Feedback via Adaptive Pruning for Code RL in GPU Kernel Generation
- Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models
- Hi-TTRL: Regulating Consensus with Hints for Test-Time Reinforcement Learning
Source: arXiv cs.CL | 2026-09-04