you can condense long horizon evals with agents into smaller subsets that still let you test intended behavior. i'm currently evaluating an …

you can condense long horizon evals with agents into smaller subsets that still let you test intended behavior. i'm currently evaluating an agent that runs for 30+ minutes, and analyzes thousands of traces at a time. here's my process: if you're evaluating whether X impacts Y agent output, oftentimes a lot of the information that exists in X isn't relevant to the decision of Y. i extract the reasoning out of trace, and then figure out what is the cause of a specific behavior. then, i know what situation i need to re-create when setting up my eval. and as a result, i can create a much smaller/simpler version of the long horizon eval that i can quickly use to figure out what i need to change in my prompts to get the behavior i want.

Source: Harrison Chase (X) | 2026-05-20