EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agents

arXiv:2601.16690v2 Announce Type: replace Abstract: We introduce EMemBench, a programmatic benchmark generator for evaluating long-term episodic memory of agents through interactive games. Rather than using a fixed set of questions, EMemBench generates questions from environment-grounded trajectories, covering both text-only and visual game environments. Each template computes verifiable ground truth from underlying game signals, with controlled answerability and balanced coverage over memory skills: single/multi-hop recall, induction, temporal, spatial, logical, and adversarial. We evaluate memory agents with strong LMs/VLMs as backbones, using in-context prompting as baselines. Across 15 text games and multiple visual seeds, results are far from saturated: induction and spatial reasoning are persistent bottlenecks, especially in visual settings. Persistent memory yields clear gains for open backbones on text games, but improvements are less consistent for VLM agents, suggesting that visually grounded episodic memory remains an open challenge. A human study further contextualizes the difficulty and interpretability of EMemBench.

Related

• LongMemEval-V2: Evaluating Long-Term Agent Memory Toward Experienced Colleagues
• From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents
• Momento: Evaluating Persistent Memory and Reasoning with Multi-Session Agentic Conversations
• AdaMem: Adaptive User-Centric Memory for Long-Horizon Dialogue Agents

Source: arXiv cs.CL | 2026-09-01