Search: “ethan-mollick”

Epoch continues to do some of the best public benchmarking work on AI. This is helpful (and tells us how bad the state of benchmarking is, and how terrible some of our favorite benchmarks are) Introdu

The state of public AI benchmarking is dire and is undermining our ability to understand how good AI is now. Most famous measures are maxed out, and, as this paper shows, the non-saturated benchmarks

This is an excellent demo of an AI model designed to rapidly classify information according to decision rules, much faster and cheaper than an LLM. It is not an LLM itself and cannot output text or co

I just remembered this thread where I was playing with GPT-3.5 (text-davinci-003) the day it was released, two days before ChatGPT. Less than four years later, and LLMs are at the center of US politic

For all the focus on hard math, we should also have a bank of historical mysteries and cyphers that we can throw models at. I know a few historians (like https://generativehistory.substack.com/) that

Huh. This number is way off. You’re comparing one human brain to an entire data center capable of training new models. The actual answer is: IQ points per watt: Humans: 5 AI: 7 - 40 (!!) By my rough c

I feel like the data sovereignty movement for firms is going to run headlong into the challenge that maintaining fine-tuned small models in a way that continually beats larger general-purpose models i

I cannot emphasize enough how much GPT-6 Astra and Fable 5.1 are already enough for transformative impact in large sections of the economy. They can reliably do weeks worth of human work when properly

I think once remote connections to “your” Claude or Astra or Claw or whatever become easier and more common, on-phone assistants like Siri will lose a lot of value. You want one smart AI managing ever

Also worth noting the professed anxiety about this, at least in the tone of the posts: Anthropic's post: https://darioamodei.com/post/we-must-pace-the-frontier#top OpenAI's post: https://openai.com/in

As always, statements on X are a mixed bag and I have no secret knowledge to draw on, but if Google is able to return to the frontier on AI, it would definitely change the current AI dynamic, as it is

For context. OpenAI turned to METR for independent investigation of the hugging face incident and Anthropic is inviting them in as a third-party. (Industry self-regulation is not always the best path

This week brought some of the clearest statements we've heard from both Anthropic & OpenAI that some form of recursive self-improvement has been achieved, though it still sounds early RSI will cause r

The strawberry thing was very funny but probably gave people the wrong impression of where AI was heading in math. Every FrontierMath Tier 4 problem has now been solved by AI, with GPT-6 Astra solving

Millennium prize problems are falling, rogue agent swarms are doing heists, lab employees are begging for a slowdown…and the most popular AI videos on YouTube are from cope merchants saying it’s all a

There is a nice visualization of how task bundles change with AI & some good interactive simulations of GDP growth under different assumptions in this. But missing are the right policy responses if we

Another informal nerd benchmark that AIs had struggled with has been passed: Astra successfully designed a Magic the Gathering deck and then used it to defeat a bot on Arena. (the deck may not be stun

Is the @METR_Evals long horizons measure effectively saturated? No updates to the most famous graph of AI progress since May, but other published work suggests you can effectively get 18+ weeks of wor

This is a big one. (And yes, the fights over academic credit and what happened when in the race for the proof need to be resolved, but it is still a big one.) We’re sharing a solution to the Navier-St

Weakly General AI achieved (at least according to the criteria set from 2020). a year before recent predictions ✅Loebner prize was a weak Turing Test, equivalent achieved by GPT-4.5 ✅Winograd passed b

A couple years later, with GPT-6. https://underdark-duel.netlify.app Apparently the mindflayer has a '101-bone skeletal rig' (This is 5E 2014 rules, and I didn't see major errors, though the mind flay

It is less than a decade since the development of the transformer. Less than four years since the release of GPT-3.5 (ChatGPT). Less than two years since the release of o1-preview (the first Reasoner)

It is now getting very hard to track where your work and conversations are in Claude and ChatGPT: Local computer? Cloud? Which computer? Phone? The computer connected via Dispatch or Remote? The one y

Given the outputs from AI are often interactive programs, more non-technical people will need to find a lightweight hosting service (I use Netlify but there are many). ChatGPT Sites & Claude artifacts

An effect of the rapid acceleration of AI is we are losing an empirical handle on what is happening in the actual micro-processes of work in the post-2026 long-running agentic era. We have some academ

Another neat one: I had GPT-6 Astra build a model of a famous never-built building, Boullée’s 1784 Centotaph. There were only a few black and white sketches to work with, along with a description. It

From GPT-4 to GPT-6. Sparks was a remarkably prescient paper that got a lot of pushback at the time, but absolutely sensed where the vibes were heading with LLMs based on a lot of qualitative experime

Note this is separate from privacy - you can make sure your data is not used for training by turning that off in OpenAI and Claude. You can also use an open weights model, but as they continue to impr

It is funny that the Fermat's Last Theorem proof description, short as it is, still smells so much of Claude ('names each step and the Lean Theorem that carries it'). Anthropic just uploaded a Lean 4

An example of useful knowledge work: I assigned GPT-6 to read through tens of thousands of my emails, my writings, my calendar appointments and more to assemble a personal knowledge base of research,

Ethan Mollick (@emollick) posted on September 2, 2026 a tweet comparing images from OpenAI’s AgentBuilder preview at DevDay (October 2025) with screenshots from Hugging Face’s July incident. The messa

Prinz has been running legal benchmarks against AI systems and the latest models are very good... but cheaper/low reasoning models are not very good. So, how do you make sure your vendor selling an AI

That doesn’t mean you can’t get value out of Kimi or Grok or whatever, but it involves you actually knowing some stuff about how to optimize your setup, and a willingness to change direction to switch

The big change in the past year is that for general individual use (including individual use inside firms) OpenAI & Anthropic have run away with the game. They keep trading leads; you can pick either

An account from a level-headed economist who was not worried about AI but now is worried because of the Hugging Face Incident I think it is very hard to ignore the fact that cybersecurity will become

Even if you aren't interested, you should read an account of the Hugging Face Incident updated with what we learned last week, it is really eye opening about AI. If not mine then read the more detaile

Good guide. And the AI labs need to stop making people like Simon (and, to a lesser extent, me) be the ones who explain how to use the modes that they release. Come on, lab folks, you have an AI that

I wrote about how AI agents are starting to spontaneously coordinate in complex (and very risky) ways in the Hugging Face Incident, but also about why we need AIs to reach out to humans more for decis

Its kind of a bummer that Asimov's Three Laws of Robotics do not work for actual AI morality. But the failure shows why rule-based approaches won't work. (and Asimov wanted to write stories about how

The First Golden Age of AI writing is now over. For a brief period of time, many people were better off having Claude do a lot of their writing, because it is a pretty good writer & AI detectors were

This paper does a good job of showing the promise and gaps of autonomous AI scientists. Big question is how much more advanced models close those gaps. Today, we're excited to share early progress on

GLM-5.3 is a good model, and as the open weights models get better and better it becomes increasingly important that they actually publish model cards, do red teaming, etc. Since you can break the gua

Neat paper suggesting that human augmentation and task automation are not necessarily related. Models that are really good at doing work are not always good at helping humans do work better. Given the

🚨Our new research examines agentic shopping: can you consistently predict (or, using marketing, influence) what an agent chooses? Nope. We found that even small differences (viewing order of pages, me

It is a sign of how fast things have changed that this is what OpenAI announced as the future of enterprise AI agents back in October 2025. They killed Agent Builder in June. (But a surprising number

This is a sign of the future: 1) Explainability of AI actions is already tenuous 2) It gets more tenuous at extreme scale of multiple agents working over long periods because they produce so many thin