Knowledge catalogue
If an agent saves an engineer 2 hours to implement a feature but collectively costs the team 3 hours in alignment and review, the ROI is negative. Isolated chats are suboptimal in team environments be
On October 11 2026 François Chollet announced that he has been experimenting with large‑language‑model post‑training techniques using KerasHub, JAX, and TPUs. He indicated that portions of his work ma
The real world abounds with recursively self-improving systems, but one in particular deserves attention: Science, modeled as a system (perhaps even as an agent, with goals and resources). If you want
Announcing ARC Prize Research Summit 2026 Research Speaker Kevin Murphy @sirbayes is a research scientist at Google DeepMind and author of the “Probabilistic Machine Learning” textbooks, developing AI
Side note. The idea that current models are conscious is absurd and downright offensive. Do not confuse competent information processing and sentience, they are wholly unrelated. You can be entirely i
The critical distinction between base LLMs (2024 and earlier) and modern LRMs is not symbolic tool use. It's the switch from a transductive paradigm (intuit the answer to the query) to an inductive pa
Lots of discussion out there about our next model(!), so I wanted to give an early look as soon as possible. Introducing Gemini 4 Argon! It shows frontier performance in complex workflows, cyber defen
What is a way to test human-level generality in artificial intelligence? 'The meta-benchmark of being able to pass ARC-AGI-(n+1) immediately upon release' @Jabaluck @DAcemogluMIT The meta-benchmark of
GPT-6 Sol from @OpenAI on ARC-AGI (Verified): - ARC-AGI-3: 4.6%, 5.6K (standard harness), 23.0%, 8.7K (provider adapter harness) - ARC-AGI-2: 89.6%, 0.44/task - ARC-AGI-1: 95.5%, 0.14/task Sol reaches
I think 'AI welfare' is morally obscene (given all the human suffering in the world), and will add to the growing anti-AI backlash. AI is a technology that humans created that should be used for our b
At some point in evolutionary history, matter learned to feel. We don't know how. It wouldn't be scientific to dismiss the possibility that it could happen again. (I don't believe it happens in curren
Excited to announce that the Korean translation of 'Deep Learning with Python (3rd Edition)' by @fchollet and @mattdangerw is published! 🚀 Updated for Keras 3, TensorFlow, PyTorch, JAX, and Generative
> One could even define intelligence as the efficiency with which one converts experience into competence; by this definition they lag very far behind us Yes, one could define intelligence in this way
About a year ago, before it was on anyone's radar, we began exploring the idea of a benchmark for open-ended invention. Since then, we've developed several promising directions that will serve as the
If you ask a coding agent to write a program that maps x to y given some (x, y) examples, you're doing a very expensive form of symbolic learning. In fact what you're doing is specifically a form of D
Machine learning doesn't end when the model is trained. You still need to evaluate it, deploy it, and build the infrastructure to keep it working in production. Today's Deal of the Day takes you acros
Anthropic says its first Fermat runs failed not on math, but because dozens of Claude agents lost track of shared project state. They only succeeded after moving to Prove2Me's DAG of statements and pr
Benchmarking AI systems is a continual process that co-evolves with the models. New benchmarks challenge AI capabilities with emerging questions to shape the directions and feedback signal of the rese
GPT-6 Astra by @OpenAI achieves SOTA on ARC-AGI: - Astra scores 63% on ARC-AGI-3, 99% via a new provider adapter harness - It surpasses human performance on 96% of ARC-AGI-3 levels - It builds the mos
It is inevitable that all AI will converge towards symbolic learning (i.e. modeling data by finding the shortest symbolic program that explains it), since that is the optimally efficient form of AI. B
Side note: when we released ARC-AGI-3 in March, and frontier models scored <1% on it, a few Singularitarian poasters took it as a personal insult, and got very worked up about it. They argued the benc
Introducing Gemini 3.8 Flash, another jump in Gemini's agentic + coding capabilities, and our 3rd updated Flash model in only 6 weeks... This model has been a ton of fun to work with, excited to see w
During v3 testing for Fable 5.1 many of our requests were getting rejected due to Anthropic's safety guards misclassifying them as 'reverse engineering' We were in contact with the Anthropic team the
One way to think of current AI is as a big sponge for patterns. It absorbs and operationalizes any pattern it is exposed to (albeit with very low data efficiency at training time). Once you can progra
This is very nice work from NVIDIA. Like all high-performing approaches on ARC-AGI-3, it uses deep learning-guided on-the-fly synthesis of symbolic world models, i.e. navigating the world by generatin
Jeremy's excellent work here is a great illustration of a very powerful type of approach: LLM-guided on-the-fly synthesis of a symbolic world model, i.e. making sense of the world by writing executabl
Regular reminder -- the set of public ARC 3 games is called 'demonstration set', not 'eval set' nor 'training set'. It is not meant to be used as training data, and it is not meant to be used as an ev
Test-time training was popularized during the ARC Prize 2024 competition, after being explored in particular by @MindsAI_Jack and team. To date, I believe ARC 1-2 are the only datasets where TTT stron
Thanks. The fact that it works so well for ARC is also a clue to the extent of generalization in these models. It was not at all obvious that it would work originally because the model is never traine
The way most people leverage test-time compute today is via a form of test-time NL reasoning that is computationally equivalent to test-time search (sometimes with a verifier / grader in the loop). Th
Expedia recently moved its ranking models to a state-of-the-art Keras 3 setup. Results: 30% faster training, and inference latency decreased by 70%. Read their writeup about the upgrade: https://mediu
Coding isn't yet another application domain -- it's the meta-skill required for AI to automatically develop its own training material, via symbolic world models. That's how the RSI loop actually kicks
François Chollet commented that claims the demise of Google were greatly exaggerated, cautioning against undervaluation. According to a Polymarket report, Sergey Brin is expected to take direct oversi
In the era of base LLM scaling (2022-2024), I believed the LLM line of research would reach a capability plateau (as later seen with base LLMs). In late 2024, after the o3 test-time compute demo, I ch
Outstanding cost-to-performance from DeepSeek GPT-5.6 Luna (Max) performance for a 1/4th of the cost on ARC-AGI DeepSeek V4 Flash from @deepseek_ai on ARC-AGI (Verified): - ARC-AGI-2: 61.4%, 0.04/task
This might be a good time to mention my recent keynotes titled: 'Agentic AI is Neurosymbolic AI' I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known a
With agentic AI, workflows are increasingly CPU hungry. The share of cognition moving to the CPU keeps increasing. Scoop: AWS engineers have been told to conserve CPU compute to make sure the cloud gi
An accurate characterization of the arc of AI is that it is shaped by two trends: 1. Moving more and more logic to a neural model for tasks where training data can be densely sampled (e.g. the shift f
For a very long time most high-performing AI models were end-to-end neural models; vector input -> vector output, with only ultra-thin symbolic preprocessing and postprocessing layers (e.g. label deco
I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known as a 'harness'), running at inference time, orchestrating thousands of calls to a neural network f
Outlook where frontier AI is headed next 18 months: The AI reasoning training + harness loop works if you can produce enough data and reasoning traces (via verifiers). Proven with code and math result
We re-tested GPT-5.6 Luna from @OpenAI on ARC-AGI (Verified) following its recent 80% price reduction: - ARC-AGI-2: 59.6%, 0.18/task - ARC-AGI-1: 90.7%, 0.07/task The new results match Luna's original
To address the limits of deep learning and avoid stalling, the field of AI started by applying patch (1), which started being demoed 9 months later in December 2024 and has now become completely ubiqu
Quick reminder of what's ok vs not ok with harnesses used for playing ARC-AGI-3: 1. Not okay: harnesses that were custom-made to solve the benchmark or that contain knowledge about the benchmark forma
One of the least explored questions in AI is how to build rich open-ended environments for learning. Benchmarks evaluate intelligence, environments shape it. iLands is interesting because it doesn't j
Really interesting concept -- instead of optimizing against static benchmarks or internally generated reward models, agents are evaluated through real economic interactions. Using an external market a
Super impressed with what Harvinder and Suman have built at @airtap_ai. They've essentially turned SMS into a headless agentic execution layer for your mobile apps. You just text it to run errands, an
Standard RL benchmarks are episodic and stationary, so they don't capture the the characteristics of real-world deployment. Morpheus is a new benchmark for continual learning that provides persistent
François Chollet observes that agentic coding systems have made dramatic progress over a six-month period, representing a significant shift in the landscape of AI-assisted development. The post reflec
GPT-5.6 Sol achieved a breakthrough by becoming the first verified frontier AI model to surpass performance on ARC-AGI-3, scoring 7.8% and setting a new state-of-the-art benchmark. ARC-AGI (Abstractio
While the writing style of LLMs is still as recognizable as ever, a new trend is that humans have started organically writing like them, too (which makes sense: of course you would end up imitating th
ARC Prize 2026: ARC-AGI-3 Milestone #1 Winners Congratulations to the three winners who open-sourced their top-scoring ARC-AGI-3 solutions: 1. @tufalabs - 1.21%, 25K 2. Reki - .867%, 7.5K 3. Md Boktia
Francois Chollet argues that reality operates according to principles that can be understood and manipulated through modeling, suggesting that comprehending the underlying structure of any system enab
Marginal cost is not a detail. It is the whole thing. Imagine going to an airline, and they tell you, 'yes, we can get you to 75% of these top destinations' -- great, but how much does it cost for eac
Reporting benchmark results as a single scalar percentage is insufficient for meaningful evaluation of model performance. Comprehensive benchmark reporting should include efficiency metrics alongside
Francois Chollet proposes a speculative future concept where 'Latent Space Archaeologists' will study the trained weights of 21st-century AI models as historical artifacts to reconstruct extinct knowl
This post likely reflects on American ideals of opportunity and possibility, discussing the cultural narrative that America enables people to pursue diverse ambitions and achieve goals regardless of b
That's a great speech that captures the essence of American exceptionalism. It is not based on innate superiority, power or wealth, but rather on the fact that the outcasts of the world could set foot
Francois Chollet argues that information and energy are the fundamental long-term constraints on technological and civilizational progress, rather than physical materials or other resources. This pers
In retrospect... Yep One very important thing to understand about the future: the economics of AI are about to change completely. We'll soon be in a world where you can turn test-time compute into com