Super interesting paper from Meta. Long-horizon research agents are coming. But one common problem with research agents today is the lack of…

Super interesting paper from Meta. Long-horizon research agents are coming. But one common problem with research agents today is the lack of originality and how to decide what experiments are worth exploring. An AI research agent can propose far more experiments than it can afford to run, so the problem is not idea generation, it's deciding which candidates get GPU time. AI Research Preference Models is trained to predict which candidate solution is most promising before any of them execute. Two variants, both built on frozen pretrained LLMs. An inference-only model reasons over candidate plans, code, and previously executed solutions. An agentic model additionally runs small-scale pilot experiments before committing budget. Dropped into the AIRA-dojo agent and measured on AIRS-Bench, average normalized score moves from 0.684 to 0.711 and 0.729. Both variants reach the unguided agent's 24-hour performance in roughly 15 hours, using less than two-thirds of its execution budget, and together set new state of the art on two AIRS-Bench tasks. Paper: https://arxiv.org/abs/2608.13940 Chat with Paper: https://academy.dair.ai/papers/ai-research-preference-models-2608.13940

Related

• New research from Meta. Agent harnesses are still mostly authored by hand. This makes it hard to tune robust agent harnesses for long-horizo…
• Long-horizon AI research agents are mostly a state-management problem. It is not enough for an agent to reason well in the next turn. ML res…
• Here is a very common problem when building complex agents. Long-horizon agents (in particular) fail in two ways: the decision-maker can't d…
• Great paper from Google and colleagues. It proposes an interesting approach to improve agents on long-horizon tasks. Long-running agents slo…
• How to Run an Autoresearch Workflow with RL Agent Skills and NVIDIA NeMo

Source: DAIR.AI (X) | 2026-09-01