Model Releases
“The expected value of a fully LLM-generated, non-human-reviewed patch is a net-negative by a considerable margin.”
“The expected value of a fully LLM-generated, non-human-reviewed patch is a net-negative by a considerable margin.” 🦔AI can only fix security vulnerabilities 26% of the time. Researchers at 1Password
“The expected value of a fully LLM-generated, non-human-reviewed patch is a net-negative by a considerable margin.” 🦔AI can only fix security vulnerabilities 26% of the time. Researchers at 1Password ran over 6,000 AI-generated patches using Claude and ChatGPT against real vulnerabilities. Half the time the AI failed to fix the original bug. 4.5% of the time it created a brand new vulnerabili…
Related
- 1. Agreed w @scaling01 that Mythos appears to be better GPT 5.5 on many metrics. 2. Mythos is definitely a major wakeup call wrt security, a…
- Very interesting evaluation from the UK’s AI Security Institute of the not yet publicly available Claude Mythos Preview. On the happy side, …
- The Balkanization of Execution-Security Research for AI Coding Agents: Isolation, Access Control, and Time-of-Check-to-Time-of-Use Vulnerabilities
Source: Gary Marcus (X) | 2026-09-06