Switch Attention: Towards Dynamic and Fine-grained Hybrid Transformers
arXiv:2603.26380v2 Announce Type: replace Abstract: The attention mechanism has been the core component in modern transformer architectures. However, the computation of standard full attention scales