Through the Bottleneck: How Multi-head Latent Attention Separates Content from Position in Language Models
DGX agentarXiv:2607.23054v1 Announce Type: cross Abstract: Multi-head Latent Attention (MLA), introduced in DeepSeek-V2, compresses key-value pairs through a shared low-rank bottleneck (cKV), achieving 81% KV-