OSDN: Improving Delta Rule with Provable Online Preconditioning in Linear Attention
arXiv:2605.13473v1 Announce Type: new Abstract: Linear attention and state-space models offer constant-memory alternatives to softmax attention, but often struggle with in-context associative recall.