Model Releases
Introducing Benchmark Reviews: our new initiative to audit AI benchmarks. We are launching with 15 benchmarks: 4 Verified, 9 Flawed, and 2 w…
Epoch AI has launched “Benchmark Reviews,” an initiative to evaluate existing AI benchmark datasets. The program initially audits 15 benchmarks, categorizing them as 4 verified, 9 flawed, and 2 lackin
Epoch AI has launched “Benchmark Reviews,” an initiative to evaluate existing AI benchmark datasets. The program initially audits 15 benchmarks, categorizing them as 4 verified, 9 flawed, and 2 lacking sufficient information for a review. This effort began on 17 September 2026 and is part of Epoch AI Research’s efforts to improve benchmark reliability.
Related
- Benchmark Scores Are Pipeline-Dependent: A Reliability Audit of Cybersecurity LLM Benchmarks
- A Multi-Probe Audit of Clinical-Interview Depression Detection Benchmarks
- Forensic Reproducibility Audit of a Radiology Vision-Language Model Benchmark: From Intended Protocol to Released Artifact
Source: Yohei Nakajima (X) | 2026-09-17