ActKV: Efficient LLM Agents through Action-Guided KV Cache Management

arXiv:2609.31395v1 Announce Type: cross Abstract: Agentic LLM inference accumulates long KV caches across iterative observation-reasoning-action loops, imposing substantial memory overhead and limiting serving throughput. Existing compression methods emphasize overall output quality, overlooking the asymmetric importance of actions in driving task progress. Our key idea is to establish a compression criterion that values KV entries by their contribution to action generation and prioritizes action quality. However, iterative execution, dynamic memory demands, and scattered action-critical entries pose challenges to eviction policies, budget allocation, and paged memory integration. To this end, we propose ActKV, the first KV cache compression framework tailored for agentic LLM inference. (i) Action-oriented KV cache eviction exploits stable action access patterns to retain entries critical to future actions, supporting reliable task progress under compression. (ii) Confidence-driven adaptive budget allocation uses LLM's intrinsic confidence to adapt the budget to evolving action-critical memory demands. (iii) Page-aware compression management standardizes compression into three primitives with customized kernels, realizing practical throughput gains. On long-trace tasks, ActKV retains an average of 98.53% of FullKV's accuracy with only 25.98% of its peak KV cache memory. It also achieves 3.97 times and 3.58 times FullKV's token and task throughput, delivering state-of-the-art performance.

Related

• MemForest: An Efficient Agent Memory System with Hierarchical Temporal Indexing
• MESA:Task-Adaptive Multi-Structure Evidence Selection for Long-Horizon Agent Memory
• RecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents
• Beyond Memory: A Transactional Continuity Kernel for Long-Lived AI Agents
• HMARS: A Hierarchical Multi-Agent Memory System for Long-Context Reasoning

Source: arXiv cs.AI | 2026-09-28