I want to have this in writing so I can refer back to this tweet before it inevitably becomes a consensus view on twitter and say 'I told yo…

I want to have this in writing so I can refer back to this tweet before it inevitably becomes a consensus view on twitter and say 'I told you so!' - The mass majority of researchers and academics in deep learning not only do not understand the statistical nature of the field, but are completely clueless as to where true intelligence will come from. AGI, and indeed any general intelligence will not come from deeplearning for the same reason that being able to extrapolate a 4th point from a set of 3 points you've interpolated is impossible - extrapolation from solely interpolation over a bounded domain is impossible. @GaryMarcus put it best in his 2018 work (which I highly recommend you read here: https://arxiv.org/pdf/1801.00631), Deep learning fundamentally lacks the framework necessary to learn abstractions. AI as we know it today is at the very core, simple polynomial regression over a known data set that we approximate over. To ever go beyond this, you either a) must continually expand the data set that you interpolate over (which is commonly referred to as the notion of 'continual learning') or b) discover a new paradigm in which abstractions are genuinely learnable and understood by whatever system is encountering them. That being said, AI I argue is now going through a dark age. Much to the same way symbolic AI under the actions of Marvin Minsky sucked all the air out of the room and left connectionism and all other paradigms of AI dead, Deep Learning is now doing the same. All new startups, academics, researchers, etc - are all converging on the exact same approach. Take a deep learning network, and either scale it (like we've seen with the embarassingly bad performance of 'just scale a VLA!' from a number of high profile labs), or try to simply attach a harness around deep learning network and pray it all goes away. Genuine innovation in the field has more or less completely been pushed out by people attempting to make mediocre increments on a dead approach simply because it less scary to do so, and because observing marginally better improvements is more calming than going through the wilderness of trying to discover a real solution that most people are unaware of. Evidence to the above is obvious. By now, we've seen over a dozen startups every quarter come out with ludicrously large rounds with the operating philosophy of 'just throw data' or 'talent' at the problem and known solutions but in time I predict that most of these companies will not only have failed, but likely have produced little of substantive value. The amount of papers in NeurIPS for example that are all essentially making the same claim of 'here's how we hyperparameter tuned to this specific task and observed a 10% increase on this benchmark we gamed over SOTA' has become so high, that finding actual novel work in the field has now become an extremely difficult task. To that end, I'll also close off this short essay by clarifying, I don't think deep-learning is useless. LLMs and indeed transformers, CNNs, LSTMs, and other architectures have been used to do very useful and tangible things, whether its write good code, recognize fraud, etc. I do not mean to insinuate that deep-learning is useless, but the best analogy to give is everyone is scrambling to get flight, but we're all focused on building rocket engines instead of wings. With @RichardSSutton's recent announcement, I felt even more inspired and vindicated in my viewpoint. I've long held this view, and have been called crazy at every step of the way. But now as more people within the field cannot deny the ugly truth we face ourselves with; I increasingly receive more messages from friends whether researchers or founders all saying the same thing 'you were right.' Of course, like most - we (the company we've founded) believe in a particular approach that will work based on a combination of methods, namely around neuro-symbolic approaches and evolutionary algorithms. Only time will tell how right we are. The future of AI will very interesting.

Source: Gary Marcus (X) | 2026-07-14

Related