Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback

arXiv:2609.38931v1 Announce Type: cross Abstract: Self-consistency samples many reasoning trajectories and aggregates their final answers, treating the LLM as a black box that returns one answer per trajectory. Yet the final answer of each trajectory is sampled from a softmax vector that is available from the model's log-probabilities. We refer to this as the grey-box setting in which each trajectory reveals this answer distribution rather than a single draw from it. We formulate efficient inference in this setting as sequential mode identification with distribution-valued observations: sample trajectories one at a time and stop as soon as the LLM's modal answer is identified at a prescribed confidence level. We characterize the asymptotic stopping rate of mode identification with distribution-valued observations exactly and show that it is never worse than the black-box rate. We then propose the ASC-D algorithm, a betting stopping rule that attains this asymptotic stopping rate. On MMLU-Redux, ASC-D uses 46.4--95.6% fewer trajectories than answer-only adaptive self-consistency baselines and achieves the highest fixed-budget correct-certification rate across three open-source models.

Related

• VecCISC: Improving Confidence-Informed Self-Consistency with Reasoning Trace Clustering and Candidate Answer Selection
• PETS: A Principled Framework Towards Optimal Trajectory Allocation for Efficient Test-Time Self-Consistency
• Diversity Combining for Multi-Path LLM Reasoning
• TrAC: Trace-Conditioned Answer Consistency for Efficient Uncertainty Quantification in LLMs
• SliceGraph: Mapping Process Isomers in Multi-Run Chain-of-Thought Reasoning

Source: arXiv cs.AI | 2026-10-01