The real news here isn’t the result; it’s what we were not told. 1. AI once tried to be a science. Now we get stuff like the completely vagu…

The real news here isn’t the result; it’s what we were not told. 1. AI once tried to be a science. Now we get stuff like the completely vague report from OpenAI below, and a lot of ignorant questions from people who don’t know how to think critically. “Same procedure”? “using an unreleased model”? This would never pass peer review. We don’t know what the procedure was. We know nothing about the architecture (e.g., were proofs generated in one shot, and then verified by Lean? was there an iterative process?). We know nothing about the failure rate. We know nothing about the training/post training/data agumentation. 2. As a result we have zero idea of how generalizable the result is outside math. 3. A lot of X has been reduced to an ignorant cheering section that applauds without knowing what it is applauding or what it might mean — without ever asking basic scientific questions. The new system could be a legitimate step towards AGI or just a clever leveraging of Lean and synthetic data in a verifiable domain with no generality whatsoever; from this report we can tell almost nothing.

Related

• “AI: it feels most impressive when you don’t know much about the subject, don’t care or don’t have a clear idea of what the you want… Once y…
• “The mechanism is always the same in every story I've been covering. The demo works in a controlled environment with clean inputs. The deplo…
• Of course Sam is busier than ever. OpenAI is in deep trouble, and, no, Codex ain’t gonna save his company from collapse. Non sequiturs won’t…
• Was this deadpan? As if we hadn’t experienced the internet era. Let me re-up: plausible that LLMs will slow down science today while *also…

Source: Gary Marcus (X) | 2026-10-07