Search: “papers”

// State-Externalizing Harnesses // A new paradigm is emerging on how to effectively build agents and harnesses. If there is a state that the environment can maintain reliably, it probably doesn't bel

// Reusable Context Engineering // Context bloat quietly kills long-horizon runs, but you can fix it from the outside without fine-tuning the underlying agent. (bookmark this) Context management is us

Very good advice on self-improving agents. (bookmark it) This is something I am seeing in my own experiments with coding agents and harnesses for long-horizon tasks. What I have found is that stronger

// Scaling Laws for Agent Harnesses // If you build agent harnesses, this one is worth your time. (bookmark it) Most harness tuning treats every token and tool call as if volume is all that counts. Ne

AI systems may soon help run economies, infrastructure, and military operations. But these systems are not reliably loyal or secure. An adversary can make an AI work against its own operator. In our n

Our team at @AIatMeta is excited to announce ATLAS: one of the largest automated formalization efforts to date. ATLAS contains Lean 4 formalizations of both statements and proofs from 25+ mathematics

Too many business leaders believe that AI says what it means. And it’s odd because we naturally attribute a high number of human traits to AI, and yet we refuse to believe it can have hidden intent? 3

For over a decade, we’ve accepted that end-to-end backprop is the only way to train deep networks. But holding the entire network in memory all at once is why AI training is hitting a resource wall. W

Get a state of the art protein structure prediction in just 9 lines of code. No MSAs needed. I'm so excited to show the world what we've been working on the for the past months!! I'm going to highligh

Identifiability is what it means to learn the right latent space, the Gaussian is why LeJEPA works, and it's what makes planning transfer. I think identifiability is the right definition of what it me

Stronger models do not always need lighter harnesses. Everyone believes more structured harnesses universally improve reliability, and that higher-capability models need proportionally less structural

// Your Agents are Aging Too // Huh!? They need 'sleep,' and now they are aging? Joke aside, great write-up on reliable agentic engineering. This new research introduces AgingBench, a longitudinal rel

arXiv:2605.24076v1 Announce Type: cross Abstract: Modern Artificial Intelligence achieves remarkable predictive power by optimizing statistical risk functionals over vast corpora. Yet a gap separates

System scaling is the next real bottleneck in agentic AI. If you build agent orchestration layers, this is a clean map of where the engineering leverage actually sits. The labs own the model. You own

📢 Accepted to TMLR, with reproducibility certification 🏅 v2 of our JEPA-WM study (arXiv:2512.24497) is out, with new data-scaling experiments, a Lipschitz analysis of multistep rollout training, and e

// The Efficiency Frontier in LLMs // (bookmark this one) How much are you overpaying for context you do not need? It turns out that context costs dominate production LLM bills, and the right strategy

As more people come to recognize the tells of AI, which mostly happens as you start to work with AI a lot, the scales are going to fall from their eyes and they are going to realize what some of us al

Can frontier models forecast scientific progress? Mostly no, but here is why. This work looks at 4,760 scientific events across disciplines. Frontier models can identify plausible research directions

Image diffusion models like Flux natively output at 1k resolution, but what if we want to generate much higher resolution images (6k+)? SEGA modifies the RoPE encodings during the diffusion process to

Gated DeltaNet has been one of my favorite 'hybrid attention' newcomers in the good old transformer stack. Excited to see Gated DeltaNet-2. Adding it to my reading stack. In the meantime, I have a pri

Today we release a study on decoupling the benefits of subword tokenization for language model training, by simulating each suspected benefit one at a time inside a 1.7B byte-level pretraining pipelin

What we find most useful about this study is the decomposition. For byte-level pretraining, the ordering is throughput first and boundary signal second; the boundary benefit can be recovered without a

What we find most useful about CNA is that the intervention is simple yet powerful. The steering is a multiplicative ablation on a sparse set of MLP neurons, which makes CNA a clean addition on top of

just as i feared AI can be a powerful tool for opening people up to new perspectives, yet we find that people actually prefer to use “sycophantic” AI systems that reinforce their pre-existing beliefs.

There are a lot of coding and reasoning benchmarks for AI agents, but not a lot for document understanding - which is a prerequisite for all downstream knowledge work. We released ParseBench ~a month

arXiv:2605.14855v1 Announce Type: cross Abstract: Forecasting within signal processing pipelines is crucial for mitigating delays, particularly in predicting the dynamic movements of objects such as N

Open-ended coding training data may no longer be the bottleneck: AI can scale open-ended tasks—and even outperform human-expert curation. FrontierCS team is releasing FrontierSmith: a system for synth

Very thoughtful post on the role and success of open source, including potential applications for AVs and AI. Long, but worth the read. -- 'Open source is no longer just how good software gets built.

// Harnessing Agentic Evolution // Pay attention to this one if you run iterative agentic search loops. (bookmark it) AEvo splits the self-improvement loop into two jobs: > One proposes the next candi

Making humans responsible for their AI use seems like an incredibly reasonable way to address problems & opportunities in the use of AI for academic research, at least in the short term (autonomous sc

We’re training models wrong and it’s due to chatGPT. Even the modern coding agents used daily still use message-based exchanges: They send messages to users, to themselves (CoT) and to tools, and rece

Cool idea from Nous Research. What if you could speed up long-context pretraining with a subquadratic wrapper that you remove before deployment? That is the idea behind Lighthouse Attention. The metho

This NVIDIA remains the strongest platform for large-model inference at scale. Prefill/decode disaggregation, Blackwell-native quantization, custom kernels, and rack-scale NVLink turn GB200 into faste

tl;dr - let's not just remove negative behavior from models, but also add positive ones 👍 If anyone builds it, everyone thrives. Over the past decade, a lot of important work on AI alignment has focus

The inability of AI models to produce creative variation is a huge gap. The fact that they generate similar ideas limits their ability to do science & the same-y writing limits their usefulness in man

// The Memory Curse in LLM Agents // (bookmark it) Long histories apparently degrades agents as they become increasingly history-following and risk-minimizing. Across 7 LLMs and 4 social dilemma games

32,000 views for my tweet; 2.2 million for the nonsense it critiques. so typical. lies and distortions beating debunking by a factor of roughly 100. Wanna get a million views? Make stuff up. Take a ti

So many people misremember (or never read) what I said in in 2022 in “Deep learning is hitting a wall”, which was neither about revenue or AI’s potential upper limits. Rather, it was an argument that

“Combined free cash flow across Microsoft, Alphabet, Amazon, Meta, and Oracle is projected to FALL more than -70%, to ~100 billion, by the end of …. AI capital expenditure is consuming nearly every do

Badass tank that beats your sports car in a drag race Only when you drive the Cybertruck do you realize how incredible it is: a bulletproof tank that moves like a million dollar sports car! Reason for

Only when you drive the Cybertruck do you realize how incredible it is: a bulletproof tank that moves like a million dollar sports car! Reason for the angular shape is that the thick, ultra-hard stain

🧠 Introducing NeuralBench: a unified, open-source framework to benchmark NeuroAI models. v1.0: 36 EEG tasks, 94 datasets, task-specific + foundation models. MEG/fMRI ready. MIT-licensed, FAIR's Brain

arXiv:2512.07578v3 Announce Type: replace-cross Abstract: Global SHAP explanations are typically presented as feature-importance rankings, which identify variables that matter to a black-box model but

I want to tell you the story of a young woman who you have probably never heard of. Her name is Mary Anne. She was born on a remote island in Scotland, where life was harsh and unforgiving. On May 2,

Introducing nanowhale 🐳! A tiny DeepSeek model fully pretrained by an agent. Inspired by @karpathy's nanochat, we gave ml-intern the task of training a tiny MoE with all the architectural advancements

There's a clay tablet with the founding charter of a 12-partner company on it. Twelve merchants pooled 33 pounds of gold to start the firm. The contract has the partner names, the starting capital, th