Comparing Latent Concept Formation in State Space Models and Transformers via Sparse Autoencoders
DGX agentarXiv:2609.24440v1 Announce Type: new Abstract: The quadratic scaling of Transformer self-attention has driven the adoption of sub-quadratic Selective State Space Models (SSMs) like Mamba, which compr