Search: “dair-ai--x”

There is much less signal in agent leaderboards than the rankings imply. A four-facet Generalizability Theory decomposition across TheAgentCompany, tau-squared-bench, and AppWorld finds the agent main

Very interesting new paper from Microsoft and colleagues. (bookmark it) Skill libraries are used in every major harness on the assumption that more guidance is free. This work measures what a bad skil

// Actions Speak Louder Than Words // Multilingual agent evaluation compares final answers and throws the trajectory away. The trajectory fixes cost, latency, failure mode, and auditability. New resea

Four small architecture decisions can cost up to 47% of a model's long-context performance. New research from Ai2, Carnegie Mellon, and the University of Washington isolates them. Normalization, GQA,

Very interesting new work from Anthropic. (bookmark it) They evolve mind viruses, ideas that spread through a multi-agent system by getting each host to pass them on, then measure what governs the spr

Why does CLAUDE.MD keep growing? If you maintain a CLAUDE.md or an AGENTS.md, this one is worth your time. (bookmark it) This work traces why these files grow without bound. Appending an instruction i

Impressive new paper from Meta. (bookmark it) Scaling laws assume model size and training data act on loss independently. This work introduces Skaling law, which couples capacity and data through a si

// The Bitter Lesson of Tool Calling // Tool calling is a design choice, and the defaults are quietly costing accuracy. How so? New research releases a generation-spanning comparison of programmatic t

Very cool idea to have agents design complex systems by searching over the model structure itself. New research from Sakana AI introduces CEDAR, which uses LLM agents to write, simulate, and refine sy

New research from Meta. Agent harnesses are still mostly authored by hand. This makes it hard to tune robust agent harnesses for long-horizon tasks. In this new work, agents learn harness policies off

Harness choice is a big deal. So much room to advance and improve results across the board with agent harnesses. Great paper highlighting this. New research releases DataSpace, a benchmark where data

Skill libraries are shipping in agent harnesses on the assumption that writing skills down compounds. A new benchmark tests that directly. ContinualSkillBench covers five domains, each with 100 interc

Very interesting to see @JeffDean's pitch deck. Just look at those open science and engineering problems. Lots to advance there with automated ML engineering. AI for science and engineering is just ge

Building agents that patch other agents. This is an interesting approach for self-improving agents that leverages agent outputs. If you run agents in production, you already have the training data for

Picking the right agent harness is now a crucial skill for any AI engineer. Imagine using the same model, same task, and same prompt. Now move it between two agent harnesses and the cost per success c

Routing for long-horizon coding agents is a big deal. @notdiamond_ai just announced a model router that works natively with Claude Code. This is huge. It picks the model and reasoning effort before ea

// The confidence cliff in self-improving autoresearch // Autoresearch loops are still quite brittle. Here is a nice paper offering some insights into why this might be happening. Self-improving autor

Finally a good paper testing whether agent memory needs an LLM at all. Production memory stacks spend extra model calls on summarizing interactions, writing records, and reranking retrievals. Every on

// Model or Harness // Great paper if you are building with agents in production. (bookmark it) It organizes 41 agent failure modes by the interaction they originate in. Each mode gets assigned to an

Nice benchmark to measure agentic e-commerce capabilities. They ran an agent for one simulated year of e-commerce operations and it ends up with 27.3% of the money a human makes. MerchantBench is a 36

This is a wild result. Locus, the automated research system from @intology, post-trained Qwen3 base models that beat the official human-tuned Qwen3 1.7B Instruct release. SoTA on PostTrainBench! The m

Try Qwen3.8-Max on Hermes Agent and you will have to doubt on how much these open frontier models have caught up with frontier closed models. These new open models are insanely good. Meet Qwen3.8-Max:

New research from Google DeepMind. (bookmark it) SkillSmith treats model weights as an additional modality the LLM reads natively. The augmented model ingests existing prefix weights alongside rich te

If you maintain an AGENTS.md or a CLAUDE.md, this is worth a read. (bookmark it) 288 gold-test evaluated runs across Claude Code and Codex, 17 real tasks from 3 repositories, with context-injection st

// Persistent Workspaces for Long-Lived Claude Code Agent Teams // Four issues to be aware of: > Working state vanishes when a terminal closes and the team cannot be resumed. > Compaction condenses th

'Intelligence too cheap to meter' battle is on! Given that DeepSeek-V4-Flash-Preview is already great for agentic tasks, there is no doubt this new checkpoint must be an absolute beast. 20+ point jump

Neat work on long-horizon agents. Splitting a hard task across agents is typically how standard multi-agent work. The usual design lets them exchange findings only at phase boundaries, through staged

New research from Microsoft. This one is on training computer-use agents at scale. Recent pipelines generate synthetic environments in bulk, which moved the bottleneck from how many exist to what is i

Very interesting paper on recursive self-improvement. The whole stack is released. Machine learning engineering gives recursive self-improvement a concrete, executable testbed. OpenMLE is an open full

Finally a good paper testing if file-system based memory for LLM agents is worth it. First, what does this look like? Deployed agents keep long-term memory as a folder of markdown files they read and

// The agent is its own best speculator // Agents spend a large share of wall-clock time waiting on tool results. Speculation hides that latency by predicting and pre-executing the next call, but exte

We have to be careful to not offload our understanding to agents. I think there is also a good opportunity to build agentic applications that encourage deeper understanding. For example, coding agents

After a few more hours, I think I've figured out Opus 5. Opus 5 is trained to be more agentic than anything I've used. All Claude 5 models are like that. So what changes? The way to interact with Opus

Impressive paper! It's on one of the hardest tasks for coding agents today. Of course, I am talking about kernel optimization. Coding agents are usually not so great at this. Reasons: Unfamiliar low-l

On benchmarking long-context agentic instruction following. Agent benchmarks mostly reward reaching the answer. This new benchmark measures whether the agent reached it the permitted way, which is the

Super interesting new work from NVIDIA. (bookmark it) They suggest building agents as Python objects. Very cool idea and I think it could a lot with agent reliability. More below: Agent development to

// Unfolding Sub-Agents for Long-Horizon ML Engineering // Watch a single agent work a machine learning engineering task for six hours you see issues like context fills with stack traces, dead experim

New research from Meta and CMU. This one is on agentic context management for long horizon tasks. (bookmark it) Production agents accumulate context every turn. The usual fix compresses on a token thr

Very interesting paper on LLM reasoning. They find that frontier models can exhibit invisible reasoning by leveraging semantically irrelevant filler tokens. In other words, invisible reasoning can ser

Are agent skills always worth using? The answer is no? This paper provides some important insights to understand this more. (bookmark it) Paper summary: Adding procedural skills to an agent is usually

Great technical paper from Google. Great read on why context beats scale for agents working against unfamiliar APIs. (bookmark it) GPU kernel optimization has KernelBench to hillclimb on. TPUs had not

Great technical paper from Harvard and MIT. It's on role drift in compound LLM systems. (bookmark it) End-to-end RL improves the accuracy of a multi-module LLM pipeline without constraining how the mo

Kimi K3 is now available for inference & training in @FireworksAI_HQ. Crazy how easy they make it to tune frontier open models like K3 using LoRA adapters. Best time to figure out how to own your inte

New research from NVIDIA. They just dropped a PyTorch-native training framework for agentic RL. (bookmark it) Paper summary: Molt is a PyTorch-native agentic RL framework with an unusual design target

Nice little insights on doing autoresearch with coding agents. Hand a coding agent a dataset, an eval script, one editable file, and no supervision. That's autoresearch and it tries to optimize the nu

Very cool paper from Microsoft. The idea is to train agents on replayed teacher trajectories instead of live environment rollouts. On-policy distillation for agentic tasks is expensive because every u