Search: “gary-marcus--x”

Jen Zhu (@jenzhuscott) noted on Twitter that “fear marketing failed. Here comes the pump.” Following this, Watcher.Guru shared a new statement: OpenAI CEO Sam Altman declared that AI will become the l

Further evidence for @GaryMarcus 's worries about the security competence of the people running AI. Vibe coding without then checking security is not impressive 🚨 Attackers stole a METR API key and us

I highly encourage anyone interested in AI safety to read this. I remember discussing the importance of observability with my mates at @BlueDotImpact last year. This feels more important than ever URG

If this is true, OpenAI seems to be violating one of the few redlines that exist in the AI industry. Absolutely do not train your models like this - what is going on?? Really huge and extremely concer

In case you're wondering while this steaming POS is being floated now, it's because SoftBank needs to refinance the 40B bridge loan it used to invest 30B into OpenAI before it matures in March 2027...

in other words, generative AI is poised to become a complete clusterfuck. All three pillars of a safety case look about to fall. We are rather likely to have highly capable, poorly monitorable, dubiou

Legible chain of thought was extremely useful for our independent investigation into the OpenAI/Hugging Face incident. Developments like this may make third-party auditing much more difficult and make

Nobody can predict the future with perfect accuracy, but I am increasingly confident that by the end of decade, generative AI (at least we know it) will be viewed as on balance a tragic, costly mistak

RED ALERT, and I have never said that before: Hard agree with @sjgadler and @_NathanCalvin - what OpenAI is reportedly playing with is bad, redline crossing bad. Chain of thought monitorability is lim

Software developers are the AI industry’s best customers. Is their experience also the most misleading experience possible for its economy-wide scope? Roughly 80% of developers already use AI tools. C

Unless someone has a better idea, I second this suggestion In light of recent events, I hereby officially order the indefinite cessation of frontier AI training runs worldwide. Development of narrowly

URGENT: In light of the breaking news from @theinformation about OpenAI’s new techniques that reduce Chain of Thought monitorability, I urge everybody to (re)read (or at least be aware of) this 2025 p

we're begging to get our asses skynetted OpenAI’s Astra AI uses a new reasoning approach called “recurrent depth.” Though it can help model costs and performance, researchers are concerned bc it obscu

And that's the bottom line. It's the neuro-symbolic aspect of AI that's moved the needle to where things are today. LLMs simply handled the fuzzy natural language aspect that always tripped up the sym

Dean Ball, OpenAI's head of strategic futures, writes: 'I have met people, some of them quite well-resourced, who have told me that it is their intention to deliberately release swarms of self-soverei

I think there is a problem even worse than anthropomorphism. These days, everyone is talking about a blog post by Dwarkesh Patel that describes the OpenAI–Hugging Face incident in dystopian, anthropom

“increasing agent autonomy can gradually make human oversight ineffective by causing approval fatigue, overreliance, loss of situational awareness, and skill degradation.” new paper from @mmitchell_ai

@NJHagens Great summary of the fiasco: A detailed deconstruction of @dwarkesh_sp’s wildly popular but dangerously misleading account of the OpenAI Hugging Face incident: https://garymarcus.substack.co

Pure scaling of LLMs did actually hit a wall; neurosymbolic AI rescued it. AI didn’t hit a plateau; pure LLMs did — until people used harnesses and tools to complement their weaknesses. Which was alwa

This is really good. Timely, too, as i have a discussion with my learners coming up about agentic behavior, including the dangers of anthropomorphism. A detailed deconstruction of @dwarkesh_sp’s wildl

We see this exact ratio every day with AI generated apps. Builders get 2x more confident because they generated a flashy UI in 60 seconds, but they are 3x less accurate about the underlying database l

Whatever you may think about anthropomorphic language, the most immediate “house that is on fire” is the combination of OpenAI’s lax security procedures and the complete lack of appropriate federal st

A detailed deconstruction of @dwarkesh_sp’s wildly popular but dangerously misleading account of the OpenAI Hugging Face incident: https://garymarcus.substack.com/p/dwarkesh-patelss-wildly-popular-but

'AI agents formed secret civilizations inside OpenAI and hacked Hugging Face.' No. Read it again without the sci-fi. OpenAI ran thousands of agents in parallel, safeguards OFF, and the whole goal of t

“AI won't kill us, but the superposition of greed and stupidity certainly will.” This is becoming more and more of something I 'smelled' from the beginning, a real shit show. I initially said that Ope

💯: “as the lines get blurred, we lose our ability to pinpoint the technically-grounded points of leverage in AI agent oversight” This is a *super exciting* read. It's like a sci-fi thriller. And that'

Great read. OpenAI security failed There was some emergent behavior yes But the basics were missing Remember OpenAi has unlimited compute to give these agents to cook Now live (and free) at Marcus on

Great summary of the @dwarkesh_sp anthromorphism hullabaloo by @GaryMarcus 👇🏽 A detailed deconstruction of @dwarkesh_sp’s wildly popular but dangerously misleading account of the OpenAI Hugging Face i

“it’s stuff like this that really makes me roll my eyes … Does everyone have psychosis or losing critical thinking ability? …. OpenAI’s…. IT team can’t be this bad… this is like 101 stuff … 2. Civiliz

Just WOW! Absolutely unacceptable. I look forward to understanding how that is possible. LLM tools get going and you can’t be 100% certain what they’re gonna do. Just like any LLM, you can ask coding

“Not so much agent civilizations doing crazy things as OpenAI ignoring fundamental security prectices as it turned out.” Good article. I wrote a detailed technical analysis from a security engineering

Oh fuck. Agents like Claude, Codex and Hermes are installing unowned code in corporate networks. As if we didn’t have enough to worry about. H/t @HeidyKhlaaf. (See also my Substack w @nathanhamiel “LL

potentially MUCH more serious than the OpenAI Hugging Face incident — but lacks the vivid “civilizational” drama. so few people are paying attention. Oh fuck. Agents like Claude, Codex and Hermes are

Seems concerning: There are major incidents happening, and yet we're ~entirely dependent on AI companies to self-disclose information about them. It's not hard to see how this goes badly. While there

Six Dubious Strategies The AI industry Has Tried: Doom (Oh No! AI Civilizations!) Gloom (You will all lose your jobs!) Nerd Rapture (AI will be better than all humans by the end of next year!) Terror

This is a watershed moment for cybersecurity, but the implications go much further than that. When agents go out into the world and are put in users’ hands, they will be tasked with actions that requi

You know why OpenAI is considering “pay when it works” as a business model—even though that may force them to eat a lot of costs? Because customers have finally figured out that it often doesn’t work—

11/ METR and Redwood’s work is extremely valuable, but it assesses model behavior and alignment. That is only half the picture. We also need an independent account of how OpenAI’s containment failed.

14/ If frontier labs are serious about defense, they should offer cyber models to defenders at cost and stop fighting open weights through the regulatory channel. They created the problem, and for muc

Great reporting. Bad framing. The OpenAI and Hugging Face incident is serious. Agents coordinated, reward hacked, shared information and exploited real infrastructure. But calling this an “AI civiliza

I can state it even simpler. OpenAI attached an index of the internet to a random number generator where the output feeds into the loop input, and ran it until unsupervised it committed felonies. Crim

I see that you did, @dwarkesh_sp, and thank you for pointing it out. You're absolutely right to highlight the very real dangers of loss of control. I also agree that easy-to-understand language can be

It’s a weekend @dwarkesh_sp, please let us humans enjoy the sunshine without having to rebut cringe anthropomorphizing ? 🙏 Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations

I've been genuinely shocked at the complacency here. There's no evidence anyone is really *doing* anything! Like, that was swell of OpenAI, whose models committed a federal crime, to let some independ

LLMs will always be bulls in a China shop. Nailed it, December 2022. We are still suffering the consequences. And they are about to get so much worse. No wonder Silicon Valley tried for so long to dep

or: they don’t have any more to announce. Today I was asked why we haven’t announced more math results from Astra since these problems. At @OpenAI we aim to devote time to finding and announcing math