Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
DGX agentarXiv:2609.13285v1 Announce Type: cross Abstract: The KV cache is a primary bottleneck for Transformer decoding: its memory footprint and cache-read traffic grow with sequence length. Grouped-query at