Routing Absorption in Sparse Attention: Why Random Gates Are Hard to Beat
DGX agentarXiv:2603.02227v2 Announce Type: replace-cross Abstract: Learned gates can approximate sparse attention patterns on frozen transformers, yet provide limited benefit over random gates when trained joi