KVFetch: Temporal Prefetching for the Missing Half of KV Cache Compression
DGX agentarXiv:2610.08811v1 Announce Type: new Abstract: As context windows scale to tens or hundreds of thousands of tokens, KV cache compression has become essential for efficient LLM inference. Existing met