Follow the Mixed Groups: Ridge-Tracking Gaussian Sampling for Zero-Variance Web Rollouts

arXiv:2610.25330v1 Announce Type: new Abstract: Group relative policy optimization learns from a question only when the sampled responses disagree. A group that is entirely correct or entirely incorrect has zero reward variance and a zero advantage. On the Web, that group has already spent its searches and page fetches. Gaussian curricula aim the next batch at pass rates where a finite group can still mix. A Gaussian on the latest empirical rate treats every all-fail history as the same point, and a center stepped by the average signed change of those rates is pushed by prompts already stuck at 0 or 1. RIDGE keeps the Gaussian curriculum and changes the coordinate, the center, and the width. Each prompt carries a Beta posterior. The sampling weight is the marginal of a Gaussian target under a Gaussian approximation of that posterior, divided by a prediction of tool-call cost. The center and the width are an exponential smooth of the posterior means of groups that just came back mixed. All-correct and all-incorrect groups refresh the posterior and do not move the center. The GRPO update itself is unchanged. On two math backbones and a retrieval-augmented question answerer, RIDGE raises average accuracy by 3.4 and 2.9 points over GRPO and average exact match by 3.7 points, at 29.6% fewer math rollouts and 32.7% fewer tool calls than dynamic sampling.

Related

• Prompts Live on an Arc: Gaussian Curricula in Fisher--Rao Coordinates for Rollout-Efficient GRPO
• Where Groups Split: A Probe Prior for Nonzero Within-Group Variance in RL Post-Training
• BPPO: Binary Prefix Policy Optimization for Efficient GRPO-Style Reasoning RL with Concise Responses
• ThinkPrior: Zero-Rollout Difficulty Priors for Cold-Start Prompt Selection in RLVR

Source: arXiv cs.LG | 2026-10-20