PQ-HSA: Reusing Product-Quantized Scores for Hybrid Sparse-Approximate Attention
arXiv:2609.33746v1 Announce Type: new Abstract: At each decoding step a language model attends over the key-value (KV) cache of every earlier token, so at long context the attention call is bounded by