Pull: Lazy Materialization of Working Memory for Stateful LLM Conversations

arXiv:2609.14773v1 Announce Type: new Abstract: As LLM conversations grow to hundreds of turns, full-context injection incurs O(N^2) cumulative token costs, while lossy summarization or hard truncation irreversibly discards historical state. We propose Pull, a session router that maintains an addressable metadata directory via a local, deterministic Purifier (zero LLM calls, millisecond-level latency). At query time, the LLM lazily materializes only the turns it needs; unmaterialized turns remain accessible but collapsed. Unlike irreversible compression, Pull's materialization is reversible; subsequent queries can expand any collapsed turn. On LoCoEval (128 conversations, 12,780 turns), Pull reduces per-query context tokens (Phase 2) by 75.1 percent on single-hop tasks with equivalent quality (Delta = -0.002, n.s.) and by 72.0 percent on multi-hop tasks with no quality loss (Delta = +0.017). A controlled routing benchmark (7,831 queries x 10 methods) shows that entity lifecycle tracking is empirically a prerequisite for distance-independent routing. On BEAM 1M (14 conversations, 263 questions), Pull improves F1 by +55.2 percent over a truncation baseline.

Related

• LiveMem: Maintaining Memory State Continuity in Long-Running LLM Inference
• Reconstructing the Right Episode: Evaluating Interleaved Conversational Memory Beyond Long Context
• Parallel Causal Associative Fields: Gated Sparse Memory for Long-Context Language Modeling

Source: arXiv cs.CL | 2026-09-15