KVpop -- Key-Value Cache Compression with Predictive Online Pruning
DGX agentarXiv:2607.05061v1 Announce Type: new Abstract: Key-value (KV) cache growth is a major bottleneck in autoregressive decoding, as memory and bandwidth scale linearly with context length. Existing KV ev