The Last Harness You'll Ever Build

arXiv:2604.21003v1 Announce Type: new Abstract: AI agents are increasingly deployed on complex, domain-specific workflows -- navigating enterprise web applications that require dozens of clicks and form fills, orchestrating multi-step research pipelines that span search, extraction, and synthesis, automating code review across unfamiliar repositories, and handling customer escalations that demand nuanced domain knowledge. extbf{Each new task domain requires painstaking, expert-driven harness engineering}: designing the prompts, tools, orchestration logic, and evaluation criteria that make a foundation model effective. We present a two-level framework that automates this process. At the first level, the extbf{Harness Evolution Loop} optimizes a worker agent's harness H for a single task: a Worker Agent W_{H} executes the task, an Evaluator Agent V adversarially diagnoses failures and scores performance, and an Evolution Agent E modifies the harness based on the full history of prior attempts. At the second level, the extbf{Meta-Evolution Loop} optimizes the evolution protocol Lambda = (W_{H}, H^{(0)}, V, E) itself across diverse tasks, extbf{learning a protocol Lambda^{(ext{best})} that enables rapid harness convergence on any new task -- so that adapting an agent to a novel domain requires no human harness engineering at all.} We formalize the correspondence to meta-learning and present both algorithms. The framework extbf{shifts manual harness engineering into automated harness engineering}, and takes one step further -- extbf{automating the design of the automation itself}.

Source: arXiv cs.AI | 2026-04-24

Related