Predicting Future Utility: Global Combinatorial Optimization for Task-Agnostic KV Cache Eviction
arXiv:2602.08585v2 Announce Type: replace-cross Abstract: Given the quadratic complexity of attention, KV cache eviction is vital to accelerate model inference. Current KV cache eviction methods typic