Curriculum Learning for harnesses - should we teach agents how we teach kids? start small and easy and progressively get harder for my resea…

Curriculum Learning for harnesses - should we teach agents how we teach kids? start small and easy and progressively get harder for my research friends here's an under-explored area we're thinking about on how we should sample data for gradient-free hill-climbing with evals some open questions: - should we design curricula stages by category (retrieval, tool-use) or difficulty or both and what's a good sample? does it matter? - how much are learnings from evals dependent? ex: I want to be good at tool-use and reasoning before diving into agentic coding - how well does difficulty map between models? are there some universal task types that all models consider easy/medium/hard there's a lot of assumptions baked into the decision of random, stratified sampling of data for hill climbing. the mechanism of update and learning signal for harness hill-climbing is different from RL, that might mean something or nothing about data design

why not? what in the current harnesses are not bitter lesson-pilled? off top of head: will go away: - planning tools - compaction (or at lea…
a useful mental model on how teams can think about good data design to improve their models/agents: Evals ~= Training Data ~= Environments -…
https://x.com/ashpreetbedi/status/2041568919085854847?s=46 great related write up from a team that pushes taking a rigorous systems engineer…
What does it actually take to make agents better over time? A system that starts with a trace. You capture traces of agent behavior, enrich …
Another banger article from the @LangChain team! Harness evolution combined with specialist local models will be the way forward undoubtedly…
here's how we're improving our base harness, you can apply these same lessons to hill-climbing for your application-specific harness!

Source: Harrison Chase (X) | 2026-04-08

Related