Model Releases
so it was the harness not the model. more sneaky games with unacknowledged domain-specific neurosymbolic AI doing the real work.
so it was the harness not the model. more sneaky games with unacknowledged domain-specific neurosymbolic AI doing the real work. OpenAI published 99.9%. The benchmark's own software returned 62.7%. Sa
so it was the harness not the model. more sneaky games with unacknowledged domain-specific neurosymbolic AI doing the real work. OpenAI published 99.9%. The benchmark's own software returned 62.7%. Same model, same test, different scaffolding around it. ARC Prize printed both numbers and says it is not claiming AGI. https://thenextweb.com/news/openai-astra-arc-agi-3-harness-62-7-vs-99-9-benchmark-revisions
Related
- doomers and AI fans love hearing “it’s over” - but @scaling01 really should have noted that the below refers to a domain specific benchmark …
- OpenAI quietly altered the performance metrics it reported for GPT-6-Astra in ways that favored the new model, and it continues to change ot…
- The real lesson here is that if you pick and choose your stats (and which quarters to show, and so on) you can tell any story you like.
Source: Gary Marcus (X) | 2026-09-08