Search: “ethan-mollick”

Qwen 27B is really good local model but, when you use it, it is immediately absolutely and obviously nowhere near as good as the other models listed here for agentic tasks, and especially for the kind

Sovereign AI models lagging behind cutting‑edge systems risk significant disadvantages when applied to frontier tasks such as cyber defence. Ethan Mollick highlights that this penalty is most pronounc

If alignment issues are becoming big enough that OpenAI is willing to commit 20% of research inference compute to chain-of-thought monitoring, that suggests that alignment issues are becoming a pretty

AI diffusion in political campaigns is quite high (and it looks like Anthropic’s fight with the Pentagon may have been good for its uptake among politicians, through it is impossible to establish a de

Here are 22 more gems according to the AI among my bookmarks 1. Researchers compressed 4.1 million recipes into a two-megabyte model of human cooking. The model spans seven languages and represents 1,

Its interesting to see the experiments on how to give AI a computer: Codex & Claude Code use your local machine, ChatGPT Work on the web (as well as Claude and ChatGPT) give the AI a one-time machine

Here are the gems the AI picked: 1. Sperm whales may have collectively learned how to evade whalers. Within a few years, the success rate of whalers’ harpoons reportedly fell by 58%, suggesting that i

o3-mini (very obsolete) in an agentic loop made good exam questions: 'In one of the largest field studies of AI-generated questions to date, we found that this approach achieves psychometric propertie

Social science has long addressed the challenge of evaluating qualities that cannot be objectively verified—such as writing quality or creative ideas—and its methodologies remain relevant for today’s

The benchmark for non-verifiable domains is often the opinions of humans. That is how we determine whether writing or an idea or a pitch is good in the real world And we know how to measure & benchmar

I think this will turn out to be wrong, and not just because I suspect there are greater returns to more intelligent models than people expect Economic value comes from agents, not chatbots. And accur

To the extent that AI use boosts firm performance, some early signs here that early AI adopting firms that were already doing well may start to outpace others. Data from OpenAI shows some firms are us

.@BharatKChandar, @RuyuChen and I just released an updated version of our paper 'Canaries in the Coal Mine?' The headline is that we still don't see any widespread job displacement due to AI, but the

Interesting research suggests caution in determining which AI company is winning by looking at any one source.. OpenRouter seems to show open weights winning over time, but work submitted to Pangram i

Oh no, we aren’t going to go back to this sort of prompting again, are we? I would love Anthropic to test if it actually works robustly, because our experiments (with slightly older models) found it d

A failure of ChatGPT Work & Claude Cowork is they assume that non-coders couldn't understand how to think about problems like a coder, so they hide all that stuff. They should instead explain choices

You may have been told to watch this video about the OpenAI AI hack. You really should, even if you don't usually care about tech stuff. If nothing else, click this link to the 18 minutes in & see how

On August 7, 2026 Ethan Mollick tweeted that “every remaining good AI benchmark score has an implied asterisk next to it which reads: * could be significantly higher with a better harness.” The commen

It is past time to take AI & security seriously at the individual level as well. If its not the current OpenAI and Anthropic models doing it, then the coming open weights models will when they catch u

This definitely seems like something worth noting, and illustrates the gap between Fable/Astra class models and the previous frontier that was 'merely' good at hacking under human instructions. Initia

This paper by researchers from MIT and Stanford finds that most people would be financially better off if they followed the financial advice of LLMs (GPT-5.2 & Gemini 3 Flash) But some people get a bi

Also I think AISI is a great model of a government agency tasked with AI security. They have open benchmarks, very fast testing, and clear communication about incidents that is neither hyped up nor hi

Yes, the AIs were given a cybersecurity challenge, with internet access enabled and safety filters disabled. But the extent to which Mythos 5 pursued its mission (fake identities, social engineering,

Given today, it is surprising how daring Microsoft & Google were initially with AI. Microsoft released GPT-4 before OpenAI, didn't back down after Sydney & got Copilot to market quickly (the 1st profe

Nate is right (full context can lead to a degradation of chats in several different ways) but you can't ask the AI about stuff like this as they have bad self-knowledge A good approach is to compact t

Some things to note: 1) AI is getting very good at math and science. 2) Two years ago LLMs could not do basic math consistently 3) According to @polynoamial this cost less than $2000 in current API co

One of our big findings in our study at Procter and Gamble was that AI blurred the lines between jobs. Now OpenAI has a similar finding. Organizational boundaries are becoming porous, the walls thinni

This is both a real incident (in that the AI really did get unauthorized access to real systems) and also something it was (sort of) prompted to do. In a review of our cybersecurity evaluations, we fo

This is not optional, things are getting chaotic now, and not dealing with this change won't make it go away. Plus, this could be a huge boost for both individual satisfaction & firm performance if do

Answer from OpenAI @emollick re ARC-AGI-3: human testers scored ~48% (ARC uploaded testers logs to HuggingFace some time ago, I believe). re GDPval: it’s close to saturated now, so we’re mostly lookin

As the benchmarks that test frontier AI on get more complex, we are losing one of the most important aspects of benchmarking: comparisons to humans Validated benchmarks need to have human (ideally mul

Ethan Mollick highlights that large, currently under‑explored benchmarks (e.g., ARC‑AGI, GDPval, METR long horizons, ASI cyber tasks) are growing in complexity, yet they increasingly lack systematic h

Model + harness. We have barely begun to understand the best ways to do harness engineering. A huge amount of untapped potential even without models getting better (but models are getting better) Turn

There is definitely real jaggedness both among fields (writing is an area where models improve slowly if at all) and within them, but the magic of LLMs is that they are so unreasonably effective acros

There's been a talk about how LLMs are only advancing in verifiable areas like math or coding, but that isn't what the data suggests. As models have gotten better at that, they are also better at solv

We are in a world where you can create truly unique, visually interesting and creative playable demos on demand with the current capabilities of Codex and Claude Code. We don't need to keep cloning th

Ha! It did it: 'We introduce BenchBenchBenchBenchBench (BBBBB), an executable benchmark of AI-authored conformance suites for benchmark-evaluation metrics' I really thought it would treat 'now do benc

As a joke I prompted Codex 'Build and run BenchBench, a benchmark of now good ai is at creating benchmarks. then figure out what benchbenchbench is and run that. and then write benchbenchbench up as a

Glad to see Google sharing data on how Gemini is being used. Especially interesting is that the usefulness of multimodal AI for manual labor may be greater than expected. https://blog.google/innovatio

The GPT-5.x Pro series has remained the best models for hard technical problems since they launched. There is some parallel model magic going on that is not well-explained. Anthropic has never had an

This is a big jump in ARC-AGI-3. Claude Opus 5 from @AnthropicAI is the new SOTA on ARC-AGI-3: 30.2% The previous high score (7.8%) was set by GPT-5.6 Sol (Max) Throughout our analysis, we observed no

I wrote the latest of my occasional guides to which AI to use right now for non-experts who want to get stuff done. The agentic systems available to everyone are getting extremely powerful (even as th

And yes, the models were following instructions, they just did so in clever ways. Some nice additional info here A few thoughts on the Hugging Face hack: - This is, to my knowledge, the *third* disclo

“Generate a fake, but believable, witty Churchill insult at a party and explain the context. It should be very clever and original” This time, I think GPT 5.6 Sol Pro wins, but Fable is good too, and

Who would you give authorship to? The person who wrote 58 words of prompts, or GPT-5.6 Pro? Dinitz-Garg-Goemans conjecture is false. This graph theory problem was open for ~30 years. The graph below h

And now from the Chinese government side. This would be a good time for cooperation between the US and China to establish common testing/acceptance standards for new models, so at least the safety cer