Safety
The “AGI-is-near” community keeps committing the same logical fallacy over and over; I have seen it at least half a dozen times today alone.…
The “AGI-is-near” community keeps committing the same logical fallacy over and over; I have seen it at least half a dozen times today alone. Every time there’s an advance, I see the same error. Here’s
The “AGI-is-near” community keeps committing the same logical fallacy over and over; I have seen it at least half a dozen times today alone. Every time there’s an advance, I see the same error. Here’s how the fallacy works. 1. Someone pretends that all cognition is created equally. (Totally untrue.) 2. Whenever AI achieves success on some form of (fancy) cognition they want you to believe that success on all forms of AI is imminent. I include three prominent examples from today below, each promising that some grand universal can be solved (“science”, “every discipline”, “every problem [people] face in life”), simply because there was an advance in a particular domain (today, math, other times coding, etc). You don’t have to be a cognitive psychologist to realize that this inference just doesn’t follow. We all know that expertise in math doesn’t guarantee genius in all domains. We all know, for example, people who are great at math or physics or programming but struggle with writing or understanding human relationships (and conversely great writers who are weak at math), etc. It’s common to be good at some forms of cognition and not others. Expertise in one domain does not at all guarantee expertise in all or even most domains. That’s precisely why people like Gardner and Sternberg developed multidimensional theories of intelligence, why the SAT tests math separately from verbal, etc. Astra is (apparently - we still haven’t seen the methodology) great at math, or at least some forms of math, but that does not mean that it will avoid hallucinations or solve the reliability problems other GenAI systems have. It doesn’t even mean it will be able to read a PDF properly. It doesn’t mean it will be able to follow hard rules either. (Which should terrify you.) In fact, the performance we saw today doesn’t even mean it can write a decent math proof; to the contrary, the mathematician @henryquantum has already given an example where clarity in the proof was lacking. Whatever you do, please don’t get suckered into the “all cognition is alike” fallacy. It’s not all alike. Astra is obviously good at some problems; but that doesn’t mean it will be good at problems that are hard to formalize. It doesn’t mean it will be magic. It doesn’t mean it’s AGI or ASI or any of that. It’s very impressive. But I see no reason whatsoever to think it is AGI let alone ASI. If it can score even a 5/10 on my 2024 bet with Miles Brundage I will be surprised. Enough with this fallacy.
Related
- Wow this paper has been “just published” many times for nearly a year. I have called out at least two other “influencers” for the same thing…
- Crazy how many times people have told me over the last five years that a solution to hallucinations was right around the corner — and yet he…
- Every time I tell AI utopianists that biology is too complex for AI to 'solve', they cite the success of AlphaFold. No, AlphaFold did not 's…
- [[google-deepmind-ceo-sir-demis-hassabis-todays-systems-are-no|🚨 Google DeepMind CEO Sir Demis Hassabis: “Today’s systems, are nowhere near [AGI]. Doesn’t matter how many Erdős problems you solve… I thi…]]
- Everyone’s quitting but everything’s going great. AGI will be here next week. Scout’s honor!
Source: Gary Marcus (X) | 2026-08-01