Research
DCKV: Learned Transform Coding of the Prompt KV Cache in Hybrid Language Models
arXiv:2610.36950v1 Announce Type: cross Abstract: Hybrid language models keep a key-value (KV) cache only in their few full-attention layers, but this cache is still the part of a prompt's state that
arXiv:2610.36950v1 Announce Type: cross Abstract: Hybrid language models keep a key-value (KV) cache only in their few full-attention layers, but this cache is still the part of a prompt's state that grows with the prompt and that must be stored or moved to reuse it. We present DCKV, a differentiable codec for the KV cache and the first KV-cache codec that is trained end to end. Its transform, its quantizer and its entropy models, which are conditioned on the prompt tokens and on already decoded neighbouring positions, are optimized together against the frozen language model, both for the code length that the entropy models assign and for the change of the model's output. On Qwen3.5-2B, DCKV stores a compressed token in more than 55 times fewer bytes than bf16 at a loss of at most 1.8 points on three long-context benchmarks, and in more than 100 times fewer bytes at a loss of at most 3.6 points.
Related
- Thought-Aware KV Cache Compaction for Reasoning via Adaptive Attention Matching
- HijackKV: New Threat in Position-Independent KV Cache Reuse
- Comparative Characterization of KV Cache Management Strategies for LLM Inference
- Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
Source: arXiv cs.AI | 2026-10-28