Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models

arXiv:2607.22575v1 Announce Type: new Abstract: Human episodic memory supports the retrieval of experiences that unfold over extended timescales, yet the computational mechanisms underlying this ability remain debated due to the limited mechanistic accessibility in long-term memory experiments in humans. Long-context LLMs may offer promising ways to reveal plausible computational mechanisms that drive this type of retrieval. Here, we investigate whether and how LLMs capture the core behavioral signatures of episodic memory via a temporal order memory task. Using a new dataset of human behavior based on memory of a full-length novel, we show that models exhibit the same characteristic distance effect observed in humans on this task. We next apply long-context mechanistic interpretability analyses to uncover how models solve this task, and find that model performance relies on a one-dimensional temporal code that is reinstated during retrieval by a single time-reinstatement attention head. These findings support temporal context reinstatement as an important mechanism for episodic-like temporal-order memory in LLMs, offering new insights into how temporal aspects of long-term episodic memory may be instantiated in both artificial and biological systems.

Related

• Long Context Modeling with Ranked Memory-Augmented Retrieval
• Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models
• A Heterogeneous Temporal Memory Governance Framework for Long-Term LLM Persona Consistency
• TRUSTMEM: Learning Trustworthy Memory Consolidation for LLM Agents with Long-Term Memory

Source: arXiv cs.AI | 2026-07-28