Minimal-Intervention KV Retention: A Design-Space Study and a Diversity-Penalty Survivor

arXiv:2605.14292v1 Announce Type: cross Abstract: KV-cache compression at small budgets is a crowded design space spanning cache representation, head-wise routing, compression cadence, decoding behavior, and within-budget scoring. We study seven mechanisms across these five families under matched mean cache on long-form mathematical reasoning (MATH-500ite{hendrycks2021math}) with two distilled-reasoning models (Qwen-7B and Llama-8B variants of DeepSeek-R1-Distillite{deepseek2025r1}) at budgets b in {64, 128}. All seven were rejected. We then propose alpha, a one-function modification to the TriAttentionite{mao2026triattention} retention scorer that replaces argmax-top-k with greedy facility-location-inspired selection under a V-space redundancy penalty controlled by a single weight lambda. A pre-registered protocol tunes lambda on a frozen development split and confirms on a disjoint held-out split; with lambda = 0.5, alpha clears Bonferroni on two of the four (model, budget) cells (Qwen b{=}128 and Llama b{=}64), no cell is significantly negative, and the pre-registered BranchA triggers. The finding is asymmetric: a minimal scoring modification beat heavier structural redesigns in this regime, and the combined matched-memory, sympy-graded, held-out confirmation protocol is the evidence standard that made the asymmetry visible.

Source: arXiv cs.CL | 2026-05-15