OS-Marathon: Benchmarking Computer-Use Agents on Vast-Horizon, Repetitive Tasks

arXiv:2601.20650v3 Announce Type: replace Abstract: Vast-horizon, repetitive workflows are common in daily routines, e.g., processing expense reports from a stack of receipts, organising a collection of PDF annotations into structured notes, and are tedious for humans, with execution length scaling with the volume of data to process. They are, however, well-suited to autonomous agents, since their structured, recurring sub-workflows follow a logic that can be systematically learned. Yet no existing benchmark evaluates agents specifically on vast-horizon, repetitive capabilities, leaving the problem largely underexplored. To bridge this gap, we establish OS-Marathon, comprising 100 vast-horizon, repetitive tasks across 5 scenarios and 10 domains, on which our evaluation reveals that leading state-of-the-art (SOTA) CUAs struggle substantially. We further find that introducing a task orchestrator to decompose the workflow into per-instance subtasks fails to mitigate the challenge: errors accumulate and propagate across solver agents, indicating that naive decomposition is insufficient for these tasks. We then explore a cost-friendly personalisation strategy, i.e. GraphDemo, that adapts general agents to such tasks from a single human demonstration of the recurring sub-workflow logic. Extensive experiments show both the real-world challenge of vast-horizon, repetitive tasks and the improvement provided by human demonstration in this setting. Project website: https://os-marathon.github.io/.

Related

• StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents
• LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks
• What Memory Do GUI Agents Really Need? From Passive Records to Active Task-Driving States

Source: arXiv cs.CV | 2026-08-28