IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference
arXiv:2605.25475v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly expected to operate over long contexts, yet standard softmax attention incurs a KV cache that grows line