Research
What Does an Evaluation License? A Commit-Bound Census of Claim Replay in Inspect Evals
arXiv:2608.19269v4 Announce Type: replace-cross Abstract: Benchmarks can run without determining what their results license. We freeze a large evaluation collection and attempt to replay its historica
arXiv:2608.19269v4 Announce Type: replace-cross Abstract: Benchmarks can run without determining what their results license. We freeze a large evaluation collection and attempt to replay its historical claims. Most units stop because the evidence required for replay is not bound. Where replay is possible, different claims remain stable at different resolutions. We make this otherwise implicit inference step explicit and executable.
Related
- Do large language models scrutinise what they review? A multimodal audit of scoring calibration, error detection, and author-identity effects
- From Rubrics to Reliable Scores: Evidence-Grounded Text Evaluation with LLM Judges
- Qworld: Question-Specific Evaluation Criteria for LLMs
Source: arXiv cs.AI | 2026-09-01