Safety

Domesticated, not feral: Why evolvable AI is not yet a Darwinian threat Writing in PNAS (Proceedings of the National Academy of Sciences), t…

Domesticated, not feral: Why evolvable AI is not yet a Darwinian threat Writing in PNAS (Proceedings of the National Academy of Sciences), three heavyweights in AI and evolutionary biology criticized

DGX agentx-post
safetygary-marcus--x

Domesticated, not feral: Why evolvable AI is not yet a Darwinian threat Writing in PNAS (Proceedings of the National Academy of Sciences), three heavyweights in AI and evolutionary biology criticized my paper on the evolution of AI with @simonfriederich in Philosophical Studies ("The Selfish Machine?"). I submitted a commentary in reply*, now published alongside a response from the three authors. Is AI about to slip its evolutionary leash and go feral? My main thesis: today's AI is not evolving under (blind) natural selection, and won't be anytime soon. It's being bred and domesticated by us. Here's my response to their counter-arguments: 1️⃣ Many breeders ≠ blind selection. Decentralized evolution—many users, hobbyists, even some bad actors—is the historical norm in domestication. It's how we got countless dog breeds and crop varieties. When harm comes from it, it traces back to human intent (profit, malice), not blind Darwinian selection. 2️⃣ LLM "deception" is itself deceptive. It's really an impressive feat of narrative continuation, learned from human text—including text expressing all our human foibles. And labs actively select against it, making models more "aligned" (though I dislike the phrase for its binary framing). Newer models grow more docile and compliant over time. That's domestication working as advertised. 3️⃣ Their "Wuhan moment" analogy backfires—but in an interesting way. SARS-CoV-2 was forged by hundreds of millions of years of selection in the wild, whereas AI has been fully domesticated for its entire existence. So their analogy implicitly concedes that the dangerous evolutionary design work happens in the wild—not in sandboxes where humans still set the benchmarks and call the shots. So nothing to worry about then? Not quite. I think it's quite plausible that some AI systems will go feral at some point—leaking into the wild and replicating without human supervision. I share the authors’ concern about autonomously self-replicating AI agents, and I endorse their proposed safeguards. But even if some AIs were to "go feral", the likely result is a familiar evolutionary arms race between offense and defense, not civilizational collapse. Four decades of malware vs. antivirus co-evolution has never come close to disabling the whole internet, let alone threatening human civilization. We'll probably need good, docile AIs to hunt the feral ones—with neither side ever winning outright. https://www.pnas.org/doi/10.1073/pnas.2617785123 Here's their response, also in @PNASNews, I will reply to it tomorrow: https://www.pnas.org/doi/10.1073/pnas.2618094123 May be of interest to @sebkrier, @danwilliamsphil, @sapinker, @RichardHanania, @simonfriederich, @GaryMarcus, @LodeLauwaert, @tylercowen . * Who says you need an academic affiliation to join academic debates? I'm fully self-employed now. 😉

Source: Gary Marcus (X) | 2026-07-21

Loading related sources…