ClaimScope: Label-Blind Witness Verification for Mathematical Claim Auditing

arXiv:2610.21581v1 Announce Type: new Abstract: Mathematical agents are commonly evaluated by answer accuracy or proof acceptance. We introduce ClaimScope, a 72-task benchmark for auditing proposed claims through a verdict, scope, counterexample, and repair. Tasks cover optimizer-state transport, finite-horizon affine recurrences, and exact 2imes2 linear algebra. We evaluate Qwen3-8B with direct generation and four matched two-pass conditions, each starting from the same draft. A witness checker evaluates only the model's submitted state, without consulting reference verdicts or supplying a replacement witness. Its feedback still conveys information about validity: a verified refutation establishes that the claim is false, and rejection reasons can suggest structural fixes. In this evaluation, verifier feedback yields 23/39 accepted counterexamples on invalid claims and 25/72 safe completions, compared with 15/39 and 20/72 for checker-free generic feedback. Strict completion remains 17/72; verdict accuracy changes from 47/72 to 46/72. The claim-grouped safe-completion difference is .094, with a bootstrap 95% interval of [.000,.198]. All seven task-level gains follow horizon/input-length diagnostics. A post-hoc schema-only correction of the direct outputs reaches the same 25 safe tasks without another model call. These results separate claim classification from executable evidence, while locating the observed gains in witness-structure correction rather than demonstrating broader reasoning improvement.

Related

• ComBench: A Benchmark for Rigorous Proof Reasoning and Constructive Realization in Olympiad-Level Combinatorics
• Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings
• Beyond Task Completion: A Verification-vs.-Conformance Gap in Tool-Evolving Agents

Source: arXiv cs.AI | 2026-10-16