Stochastic Sparse Attention for Memory-Bound Inference
DGX agentarXiv:2605.01910v1 Announce Type: new Abstract: Autoregressive decoding becomes bandwidth-limited at long contexts, as generating each token requires reading all n_k key and value vectors from KV cach