AWM: Answerable Working Memory for Long-Document VQA Agents

arXiv:2608.25618v1 Announce Type: new Abstract: Long-document visual question answering increasingly relies on VLM agents that retrieve candidate pages, inspect page images, write findings to working memory, and synthesize answers. Working memory should carry answer-supporting evidence across page inspections for later grounded answering, yet existing evaluation mainly checks final-answer correctness and evidence-page access. This creates a memory-quality blind spot: an agent may reach the right page and answer correctly while leaving behind memory too generic or incomplete to support answering once page context is removed. We introduce memory-only answerability, a diagnostic that asks whether a reader can answer from the question and terminal working memory alone. Building on this diagnostic, Answerable Working Memory (AWM) treats terminal working memory as an answerable evidence artifact, and AWM-GRPO incorporates this signal into the GRPO reward while preserving final-answer priority. Under GRPO, this reward assigns higher advantages to answer-correct trajectories whose terminal working memory remains answerable. On extsc{MMLongBench-Doc}, even when gold evidence pages are provided, 42.5% of correct answers still cannot be answered from terminal working memory alone. AWM-GRPO improves final-answer accuracy over the RAG baseline by 8.1 and 11.9 points on extsc{MMLongBench-Doc} and extsc{LongDocURL} and reduces the memory-missing-correct rate by 2.7 points over answer-only GRPO.

Related

• V-Mem: Modality-Routed Retrieval for Long-Term Multimodal Agentic Memory
• MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA
• Trace Only What You Need: Structure-Aware On-Demand Hypergraph Memory for Long-Document Question Answering
• RippleMem: From Isolated Retrieval to Associative Recollection for Long-Term Agent Memory
• SpecHop: Continuous Speculation for Accelerating Multi-Hop Retrieval Agents

Source: arXiv cs.CL | 2026-08-27