Write What Matters: Disentangling Token Selection from Slot Allocation in Recurrent Memory

arXiv:2610.34180v1 Announce Type: cross Abstract: Long-sequence models must retain earlier information. Full attention keeps all past tokens but grows expensive, and recurrent-memory models compress history into fixed slots for linear-time processing. Limited capacity makes writes consequential: distractors can interfere with useful content, while skipped evidence cannot be recalled. Existing multi-slot memories write every token to the same number of slots, merging whether to store it with how many slots to update. We introduce Factorized Write Routing (FWR), which first keeps or skips each token and then chooses a fixed or token-adaptive number of slots. Queries still read all slots. Across ten seeds on two synthetic tasks that query facts after long contexts, we observe a density-dependent reversal. With four slots per write, long-context accuracy favors always writing on dense Feature Recall (0.472 vs 0.337), but learned selective writing on sparse-fact Passkey (0.222 vs 0.048). The tested adaptive rule underperforms fixed allocation under the learned gate on both tasks. A diagnostic oracle given fact labels unavailable at deployment reaches 0.372 on Passkey, showing headroom in filtering distractors. The results suggest avoiding missed writes in information-dense contexts and filtering distractors when they dominate.

Related

• A Hippocampus for Linear Attention: An Exact Memory for What the Recurrent State Forgets
• Proteus: Incremental Memory Activation for Long-Context Sequence Modeling
• Mamba with Hierarchical Memory: Solving Representation Bottleneck in Long Sequence Modeling
• CoTS: Competitive Token Selection for Latent Reasoning in Large Language Models
• StateX: Enhancing RNN Recall via Post-training State Expansion

Source: arXiv cs.AI | 2026-10-27