vToken: Token-Level Virtualization for Reclaimable KV Caches
DGX agentarXiv:2608.13263v1 Announce Type: new Abstract: Large language model serving faces a critical memory bottleneck: the KV cache grows with sequence length and batch size. PagedAttention uses fixed-size