Model Releases
Harnesses + smart models = the end of the last famous hard benchmarks. Models can now work for extremely long times (bye, METR Long Tasks), …
Harnesses + smart models = the end of the last famous hard benchmarks. Models can now work for extremely long times (bye, METR Long Tasks), they can output professional documents with correct math (so
Harnesses + smart models = the end of the last famous hard benchmarks. Models can now work for extremely long times (bye, METR Long Tasks), they can output professional documents with correct math (so long, GDPval) and they can solve complex novel puzzles (farewell, ARC-AGI-3). Gemini 4 Argon from @Google on ARC-AGI (Verified): - ARC-AGI-3: 97.5%, 4.8K (provider adapter harness), 49.8%, 5.8K (standard harness) - ARC-AGI-2: 91.3%, 0.32/task - ARC-AGI-1: 98.5%, 0.16/task Google DeepMind is the second frontier AI lab to score above 95% on ARC-AGI-3.
Related
- Its getting hard to benchmark frontier agent performance on longer tasks. Repeated measurement is very expensive and there are differences b…
- I find that open weights models over-perform on benchmarks compared to actual real-world usage, and Kimi feels like no exception. For exampl…
- Epoch continues to do some of the best public benchmarking work on AI. This is helpful (and tells us how bad the state of benchmarking is, a…
- Qwen 3.8 27B is now available on Ollama. It's one of the best open models at this size, and made for agentic tasks and professional work. Tr…
Source: Ethan Mollick (X) | 2026-10-15