From Interaction to Capability: Task Advancement as State and Supervision for GUI Agents

arXiv:2610.37482v1 Announce Type: new Abstract: GUI memory should preserve the results of past actions and learn reusable operations. This is difficult in long-horizon workflows, where agents must use information across applications and useful operations are interleaved with failed attempts. We introduce Task Advancement Memory (TAM), which defines task advancement as verified task progress, and supports ongoing task execution and learns reusable operations. During execution, Task Advancement State represents how verified task progress changes the remaining goals, allowing the agent to build on earlier achievements across applications. After each task, advancement supervision identifies and abstracts the interactions that produced a verified local result. The learned operations preserve the relation between required inputs, expected outcomes, and verification conditions. This retains locally successful operations from both successful and failed tasks. On MobileWorld GUI-only tasks, Advancement State alone raises success from 12.0% to 28.2%. Full TAM achieves a 17.6% relative gain in task success over the strongest evaluated baseline and reduces lifecycle tokens per solved task by 38.7%. On SPA-Bench, TAM achieves state-of-the-art task success in the single-app English Level 3 setting.

Related

• StateComp: Learning When to Compress History in Long Horizon Agents
• Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agents
• Composing Task-specific Agent Harnesses at Test Time with Reusable Primitives
• SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent

Source: arXiv cs.AI | 2026-10-28