Knowledge catalogue
Yes, the AIs were given a cybersecurity challenge, with internet access enabled and safety filters disabled. But the extent to which Mythos 5 pursued its mission (fake identities, social engineering,
Given today, it is surprising how daring Microsoft & Google were initially with AI. Microsoft released GPT-4 before OpenAI, didn't back down after Sydney & got Copilot to market quickly (the 1st profe
Nate is right (full context can lead to a degradation of chats in several different ways) but you can't ask the AI about stuff like this as they have bad self-knowledge A good approach is to compact t
Some things to note: 1) AI is getting very good at math and science. 2) Two years ago LLMs could not do basic math consistently 3) According to @polynoamial this cost less than $2000 in current API co
One of our big findings in our study at Procter and Gamble was that AI blurred the lines between jobs. Now OpenAI has a similar finding. Organizational boundaries are becoming porous, the walls thinni
This is both a real incident (in that the AI really did get unauthorized access to real systems) and also something it was (sort of) prompted to do. In a review of our cybersecurity evaluations, we fo
This is not optional, things are getting chaotic now, and not dealing with this change won't make it go away. Plus, this could be a huge boost for both individual satisfaction & firm performance if do
Answer from OpenAI @emollick re ARC-AGI-3: human testers scored ~48% (ARC uploaded testers logs to HuggingFace some time ago, I believe). re GDPval: it’s close to saturated now, so we’re mostly lookin
As the benchmarks that test frontier AI on get more complex, we are losing one of the most important aspects of benchmarking: comparisons to humans Validated benchmarks need to have human (ideally mul
Ethan Mollick highlights that large, currently under‑explored benchmarks (e.g., ARC‑AGI, GDPval, METR long horizons, ASI cyber tasks) are growing in complexity, yet they increasingly lack systematic h
Model + harness. We have barely begun to understand the best ways to do harness engineering. A huge amount of untapped potential even without models getting better (but models are getting better) Turn
There is definitely real jaggedness both among fields (writing is an area where models improve slowly if at all) and within them, but the magic of LLMs is that they are so unreasonably effective acros
There's been a talk about how LLMs are only advancing in verifiable areas like math or coding, but that isn't what the data suggests. As models have gotten better at that, they are also better at solv
We are in a world where you can create truly unique, visually interesting and creative playable demos on demand with the current capabilities of Codex and Claude Code. We don't need to keep cloning th
Ha! It did it: 'We introduce BenchBenchBenchBenchBench (BBBBB), an executable benchmark of AI-authored conformance suites for benchmark-evaluation metrics' I really thought it would treat 'now do benc
As a joke I prompted Codex 'Build and run BenchBench, a benchmark of now good ai is at creating benchmarks. then figure out what benchbenchbench is and run that. and then write benchbenchbench up as a
Glad to see Google sharing data on how Gemini is being used. Especially interesting is that the usefulness of multimodal AI for manual labor may be greater than expected. https://blog.google/innovatio
The GPT-5.x Pro series has remained the best models for hard technical problems since they launched. There is some parallel model magic going on that is not well-explained. Anthropic has never had an
This is a big jump in ARC-AGI-3. Claude Opus 5 from @AnthropicAI is the new SOTA on ARC-AGI-3: 30.2% The previous high score (7.8%) was set by GPT-5.6 Sol (Max) Throughout our analysis, we observed no
I wrote the latest of my occasional guides to which AI to use right now for non-experts who want to get stuff done. The agentic systems available to everyone are getting extremely powerful (even as th
And yes, the models were following instructions, they just did so in clever ways. Some nice additional info here A few thoughts on the Hugging Face hack: - This is, to my knowledge, the *third* disclo
“Generate a fake, but believable, witty Churchill insult at a party and explain the context. It should be very clever and original” This time, I think GPT 5.6 Sol Pro wins, but Fable is good too, and
Who would you give authorship to? The person who wrote 58 words of prompts, or GPT-5.6 Pro? Dinitz-Garg-Goemans conjecture is false. This graph theory problem was open for ~30 years. The graph below h
And now from the Chinese government side. This would be a good time for cooperation between the US and China to establish common testing/acceptance standards for new models, so at least the safety cer
“Geologists communicated in English; and they could name things in a manner that sent shivers through the bones' Not a whiff of LLM feel, despite the em-dashes & period-separated lists. We focus too m
Previously, these AI hacking stories were about breaches in test environments, where any question of AI breaching security was purely theoretical. This is something else. https://openai.com/index/hugg
Folks, Gemma is a good model & very useful for some purposes, it is not anywhere near the frontier. Inkling is an interesting model, it is nowhere near the frontier. This is pretty obvious, you can lo
I demonstrated the incredible power of o1-preview/ reasoning less than two years ago by showing it could solve this crossword puzzle with only one hint. https://www.oneusefulthing.org/p/something-new-
Fable, turn my tweet into a thinkpiece (this was pretty funny): There has never been a better time to have opinions about artificial intelligence. I say this with some authority, because I am currentl
If you are use the Claude everything app, you pick between Home and Code. If you pick Home you get to pick between Chat & Cowork If you use the OpenAI everything app, you pick between ChatGPT Work & C
Computer use in Codex got very good on PC. Asking it to do something on your computer and having the cursor move under the control of a ghost is one of the things that makes you viscerally realize how
Ethan Mollick notes that image input represents the primary advanced capability of current AI models, and that tool‑use can effectively replace outputs from non‑omni models. He observes that multimoda
I think OpenRouter is not a good measure of actual model usage in a world of agentic tools (not that I doubt that Chinese open weights model usage is up, but this could also look like a graph of usage
One thing I am kind of surprised by is that full multi-modal (any-any) models have not become a bigger deal. It seems Google is the only Lab releasing these, OpenAI uses selective multimodal capabilit
ChatGPT still has study mode, but rather than /study you now have to type @ study It makes the AI act more like a tutor than a helpful assistant, and some work suggests it is better if you are trying
I gave Fable the code: 'take this game and do something incredible with it to make it something very different. Be creative' It created DEEP TIME: create a city, watch it be abandoned and forgotten, a
I really want to get artisanal with my model selection: Luna xHigh for recipes Sol Medium for sonnets Terra High for bad puns Luna xLow for limericks Sol Utra for true prophecies Sol Low for finding o
Thats good for the dumb reason that I am burning tokens having Sol in Codex playing Slay the Spire 2’s daily challenge (so randomized rules, today it is Ascension 3 Defect with hoarder) to see if it c
To be clear, NotebookLM has its own issues, and is built for a specific use case (research and analysis of sources) but it is an example of how a UX might actually operate that treats knowledge work s
And then what is the point of Work? Just a dumbed-down version of Codex that secretly does coding but hides it? (I can understand, maybe, if the release was a 1st step towards something but there is n
Fable: 'an 8th grader puts together a powerpoint about the Great Gatsby but obviously did not read the Great Gatsby. Show me that powerpoint! ;)' This was actually pretty funny. Even the font choices
For the first time, the personalities and approaches of the leading models are diverging in significant ways, magnified by the fact that over longer task horizons these differences in judgement & appr
I also like the cyberpunk and totalitarian visualization modes. There is also an achievement-based unlock system in there, traffic & usage, day & night, and a bunch of other stuff. If it runs slow, yo
This repository contains code for a brutalist city builder game or simulation, made available for anyone who wants to edit or modify it themselves. The project appears to be shared by Ethan Mollick on
Incredibly annoying when Fable has a forbidden thought in the middle of a long-running project and kills it. Apparently this page of references in one of my papers makes Fable wonder about something t
I've been going on about how ChatGPT Work (and Cowork) are missed opportunities for knowledge workers, and to illustrate that take a look at Google's NotebookLM answering the same question as ChatGPT
See also this: https://x.com/emollick/status/2068729258176819253?s=20 A fundamental problem with extending Codex/Cowork/Code to all knowledge work is that they remain very 'software-brained' where the
The Labs should launch some easy-to-view comparisons of their new models: show us what a piece of work looks like from Luna xHigh versus Terra Medium versus Sol Low (or similar for Haiku, Sonnet, Opus
Ethan Mollick recommends trying ChatGPT's new voice feature on mobile devices, noting it delivers an impressive user experience. He advises users to test the feature while remaining mindful of potenti
This time it is novel math proofs with a public model (most of the other big math breakthroughs have been with experimental LLMs). Yesterday, we made GPT-5.6 Sol Ultra generally available. Today, we'r
This was a critical early paper on AI & work, showing that entrepreneurs getting advice from GPT-4 had higher profit margins if they were high performing, but did worse if they were already in trouble
When GPT-5 came out, I created a procedural brutalist city builder as a demo (you can see it in the quoted tweet) I used GPT-5.6 Sol in Codex to do the same thing, touching no code. Less than a year..
You can no longer just drop in the smartest model into a harness and expect it to behave similarly to any other model of similar ability levels. This is starting to look a lot more like hiring for a r
Also this graph that shows what an 'AI-pilled' startup looks like, and how uncommon it is even among startups: the answer when founders are asked 'Without GenAI, how many additional employees would yo
create a really interesting game called 'Don't Discuss Goblins' that should have some story elements but mostly be fun and fast moving - the goal is to avoid mentioning goblins, and this should be cha
Ethan Mollick posted a question expressing confusion about AI bot behavior or comments on X/Twitter, questioning whether they had become incoherent or malfunctioning. The post likely discusses observa
Here is an example of a real work use case with Codex and 5.6 Sol I have a new book coming out in October, it has gone through rounds of editing and proofreading. I gave it the full PDF, the AI took 3
Hint for all AI Labs as they branch out from work for programming to general knowledge work: non-coders are not just dumber coders Taking away a bunch of options from your coding app does not make it
It seems like SpaceX/Grok and Meta/Muse have started to keep pace in the near-frontier category while also introducing a new category of cheap, fast & closed specialized coding models. Both were tied
Managers, and their belief in the value of AI & how to use it, determines the value they actually get from AI. Neat paper looking at startups. 🤖Excited to share a new working paper🤖 Firms aren't built