pi-Attention: Online Efficient Sparse Transformers for Long-Context Modeling
arXiv:2511.10696v3 Announce Type: replace-cross Abstract: Sparse attention is crucial in long-context Transformers, which restricts each token to a limited neighborhood and thereby reduces the quadrat