Search: “ethan-mollick--x”

All benchmarks are flawed, but GPQA has been fairly consistent & highly correlated with other measured benchmars. I think it's a good way to see how far we've come that the free model from OpenAI, GPT

Also Instead of the gold standard, we can imagine an inference standard of exchange, the FLOP. (As opposed to tokens, this accounts for AI ability) With some AI help, I figure 1 buys roughly 10^17 man

Mollick argues that AI policy and development will inevitably remain contested territory because different stakeholders—including governments, corporations, researchers, and the public—have fundamenta

For individual AI use, the jagged frontier is increasingly well understood. In multi-agent workflows in organizations, AI is jagged in ways that have not been well identified yet. In fact, we don't ev

In addition to the CAISI evaluation, it would be useful if NIST conducted public tests of AI abilities as an independent evaluator - though those obviously should not be pre-release tests & can be don

Ethan Mollick advocates for dedicated funding toward developing independent benchmarking systems for AI models, noting that evaluation is currently predominantly conducted by the labs that create the

Missing from the “will AI replace doctors?” debate is that doctors (and lawyers and psychologists and bankers) all vote & form the donor base to political parties & have deep community ties. The gover

Ethan Mollick suggests that Mythos, likely an AI model or system, demonstrates strong capability in identifying and diagnosing pig diseases, possibly indicating its effectiveness in agricultural or ve

FLOPs (floating-point operations) serve as a practical metric for measuring AI computational capacity because they have a precise technical definition, can be directly priced and measured, and offer a

The unreasonable effectiveness of LLMs is what makes them so weird. The labs don’t need to decide what kind of AI to build, because better LLMs do better at most things. Finance? Pig disease identific

Too much of the solutions & language for agentic systems comes from a coding perspective (control planes, hooks, loops), but I really think that the field of management and organizations can tell us m

We know the answer: Cal State LA, followed by Pace & SUNY. State systems in California, Texas & New York are great for moving many born in the lower 20% of income to the top 20%; Ivies for moving a fe

A challenge with AI regulation and vetting is how bad our benchmarks of AI model performance and risks are. There is no benchmark for risks and red-teaming requires experiments from dedicated speciali

Co-founder of Anthropic, interesting that he refers to public sources when he is also obviously privy to lots of internal sources that he cannot discuss. I assume he sees the same thing at Anthropic.

I think the fact that GPT-4o and Llama 3.3-80B did no significant harm is just as important as whether AI helped. If older (less accurate & more sycophantic) chatbots essentially did nothing for peopl

It is somewhat comforting that now, whenever I see a post about “here’s the thing that keeps me up at night” I know that there is absolutely no chance that this is being written by a human who is stay

I cannot provide a summary for this entry as the post content appears incomplete or corrupted in the provided information. The title is cut off mid-sentence and doesn't clearly convey the full topic.

My surprise here seems warranted, this paper was retracted (There are other peer-reviewed meta-analyses of the impact of AI on education finding positive effects, like: https://www.researchgate.net/pu

Poems that ChatGPT, Claude, and Gemini all seem to 'like' when you ask for poetry related to being/making LLMs: Rilke's 'Archaic Torso of Apollo' Stevens' 'Idea of Order at Key West' Borges's 'The Gol

That doesn't mean that there should not be regulation and vetting, but it does suggest that is hard to write a criteria right now that is not somewhat vague. More R&D into non-lab benchmarks is urgent

This is a very niche post, but they are good poems. I hadn't read Autopsychography until it came up from both Claude and Gemini. And if you haven't read Rilke, that poem, with its incredible end, is a

I am not sure I would agree with all of this, but the relationship between Anthropic and Claude is quite different than the relationship between other labs and their models. And that shows up in lots

Its getting hard to benchmark frontier agent performance on longer tasks. Repeated measurement is very expensive and there are differences between using models in harnesses versus via APIs. I suspect

Of course Pynchon would call this correctly 40 years ago: “It will be amazing and unpredictable, and even the biggest of brass, let us devoutly hope, are going to be caught flat-footed.“ He and Dougla

Sometimes when I demo AI, I show it turning cover letters into goofy formats (poetry, etc) as an introduction to the idea of AI as translator between forms. For the first time, GPT-5.5 has been trying

The artificial analysis index is a normalized score of several benchmarks (and has changed over time) it is fine for roughly comparing models, it is not useful for trend analysis and it is unclear wha

The single most accurate science fiction author writing about AI turned out to be… Douglas Adams He wrote about AIs that work best when emotionally manipulated & that guilt you in turn. And he underst

This is a good explanation of why the gap between open and closed models is larger than it appears in benchmarks. I would add in that current open models are also more fragile than closed: they handle

Generally, I would say X is not real life, but I am surprised about how often I get asked by executives about which AI lab is winning or what is up with a particular model in ways that indicate that t

I was quoted a couple times in this Atlantic article, but that isn’t (the only) reason I think it is good. It lays out the reasons why we whipsawed from “AI is a bubble” to “there are not enough data

(Sorry, after seeing so many of these, could not resist): 🚨 BREAKING: Google just dropped a NEW paper that completely deletes RNNs from existence. No recurrence. No convolutions. Nothing. Just one mec

This post likely references the Voight-Kampff test from 'Blade Runner,' a fictional device used to identify replicants (artificial beings), as a metaphor for distinguishing AI bots from humans on X (f

Ethan Mollick demonstrates GPT-imagegen-2's capability to generate image grids with progressive variations on a theme, showing how the model can create multiple iterations of subjects (dogs, cats, squ

I should also add that one of the strongest alignment actions that OpenAI did was to name their product chatgpt with gpt 5.5 medium, names so uninspired that nobody could see it as a friend. Unlike Cl

I think everyone would be okay with this, though. Many people do not seem to want data centres built near them, despite the fact that they don't cause that much traffic and often generate a lot of loc

New paper (on an old AI) tests o1 against doctors on medical benchmarks & real ER cases: “across a variety of scenarios and applications, the large language model outperformed both human physicians an

Nice thread by an author 🧵1/ Our new study on AI and physician reasoning just came out in @ScienceMagazine. As co-senior author, I'm excited about our findings, and I do think AI will reshape medicine

Organizations are already superhuman intelligences. The University of Pennsylvania or Walmart or whatever is far more capable than any human. That is why the focus on AIs as individual productivity to

Randomized trial of an AI therapy chatbot on Mexican women found “improved mental health by 0.3 SD over 6 months with no evidence of an increase of severe cases; improved sleep, healthful behaviors, d

The goblin thing was fun as it was a real quirk that was emblematic of what makes AI interesting, and it organically came out of an AI user discovery. So was, for what it was worth, Ghiblitization Whe

The new Grok comes in below the latest Chinese open weights models, Grok 4 was at the frontier when released. (& Artificial Analysis: please stop using GDPval-AA which is not a useful test of anything

We need more work on AI inequality, but this study is not about GenAI, the survey was fielded in 2022. “In this study, we selected items from Wave 119 (N = 10,087), which were collected from December

This post likely discusses an unexpected or unusual infrastructure discovery, possibly referencing a road that literally terminates at a river without proper connection or explanation. Based on Ethan

This post by Ethan Mollick likely discusses how advanced AI systems like hypothetical future GPT versions could autonomously execute complex, real-world tasks with minimal human direction, using a hum

This post appears to discuss narrative and storytelling techniques that rely on implied catastrophes, unspoken tensions, and shared understanding between author and audience—elements that create drama

ASI* is GPT-X deciding I want to throw a party for myself and every human on earth will get a personalized email them that will help them in some way, and also here is a cure for a bunch of diseases *