Self-improving agents isn’t a single algorithm - it’s a systems engineering problem involving: - eval data curation + maintenance - experime…

Self-improving agents isn’t a single algorithm - it’s a systems engineering problem involving: - eval data curation + maintenance - experiment design to battle overfitting - an update algorithm - human review during the process & especially before prod we share practical learnings + a local research scaffold to autonomously hill-climb harness centered around evals our goal is to give everyone the tooling and infra to measure and iteratively their improve agents. Evals are training data for agents which fuels this loop let's build the future of well-designed, self-improving systems 🚀

tldr > evals are the new training data. instead of updating weights, you're updating the agent harness > problem is agents are famous cheate…
a useful mental model on how teams can think about good data design to improve their models/agents: Evals ~= Training Data ~= Environments -…
Agents will cheat your evals if you let them. @Vtrivedy10 wrote a great article on how we keep them from overfitting
What does it actually take to make agents better over time? A system that starts with a trace. You capture traces of agent behavior, enrich …
New Guide: Incorporating human judgment in the agent improvement loop Building agents is hard. Everyone talks about the code. What gets less…
here's how we're improving our base harness, you can apply these same lessons to hill-climbing for your application-specific harness!

Source: Harrison Chase (X) | 2026-04-08

Related