Hands on, concrete guide (with code!) for harness hill climbing with evals

LangChain's "Better Harness" tutorial, authored by Product Manager Vivek Trivedy and shared by Harrison Chase (@hwchase17), presents a hands-on, code-driven guide for using evaluations (evals) as a learning signal to iteratively improve AI agent harnesses through a process called "hill climbing." The approach centers on an autonomous experiment loop — implemented via the hwchase17/autoresearch-agents GitHub repository — where a coding agent repeatedly modifies agent.py, runs evals against a fixed dataset using LangSmith, and tracks whether changes improve performance. By keeping the evaluation harness and dataset fixed while only iterating on the agent implementation, developers can systematically optimize agents and generalize improvements beyond ad-hoc tuning.

Better Harness: A Recipe for Harness Hill-Climbing with Evals
here's how we're improving our base harness, you can apply these same lessons to hill-climbing for your application-specific harness!
Deploy a production ready agent to the web... with the same setup you use to define a coding agent AGENTS.md - open standard /skills - open …
Open swe uses deepagents under the hood Deepagents is general purpose, openswe is focused on coding

Source: tutorials

Related