Alpha Eval: Agents Making Evals as a Multi-Player Game This diagram and blurb is largely a research riff with Claude on building data genera…

Alpha Eval: Agents Making Evals as a Multi-Player Game This diagram and blurb is largely a research riff with Claude on building data generation systems to get closer to the holy grail of self-improving agents The Self-Improvement Goal & Issues: the holy grail of intelligence is recursive self-improvement where agents can take any task and improve themselves on that task without human intervention some obvious questions here are: what does this system look like? how do we encode what’s “good” for a task? Do we need Curriculum Learning to help agents progressively learn abilities? Evals & Environments are an on-ramp: My take is that Evals are our strongest tool today to start experimenting on this loop. Evals are training data for the agent world. Every time an eval fails, we update our agent definition to make it pass, that’s analogous to any RL or SFT training approach where data encodes changes to a system using some algorithm to measure and apply to change. If we can map an Ability like long horizon coding to a set of Evals that capture that ability, then it’s possible to imbue that ability into our agent Eval Generation is Hard & Curriculum Learning: But generating evals is hard. It’s close to impossible to know apriori if agents will learn the appropriate capabilities from any given eval, or if the eval design is resilient to Reward Hacking. Some ideas in Curriculum Learning may be helpful where every progressive batch of tasks are designed to train an ability that helps unlock future abilities. For example, to do Long Horizon Coding well, the agent needs to be a great Planner and Plan Updater. Curriculum Learning seeks to develop groups of tasks and give them to agents in an order where every previous task batch helps the next task batch. Tasks get harder over time. Using Trace Data: I’m very bullish on using Production Traces as a prior for good data generation. Production Traces data captures real world scenarios and actions and crucially, ground the eval generation process in a concrete signal. We’ll definitely continue pushing here because it’s the most tangible and provably good method for bootstrapping self improving data. Agent Driven Data Generation Autonomous Evals: There’s another very exciting area which is, Agents should make Evals for other agents to hill-climb on. A lot of this is inspired by DeepMind’s amazing Alpha-Go/Zero work on treating self-improvement as a game where actors use the game environment to self-improve. The diagram is largely a riff on such a system and is still susceptible to issues of eval collapse and overall it’s just hard for agents to capture good signal autonomously Though think it will be very interesting to see research groups lock in on this task of Data Generation for Self-Improvement. All of the RL Environment startups have basically made this bet that some mix of humans and agents can create great learning environments at scale. Riff over, Claude is a great thought partner for research ideas.

Source: Harrison Chase (X) | 2026-04-25

Related