Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View
DGX agentarXiv:2606.04405v1 Announce Type: cross Abstract: Modern Transformer architectures frequently employ normalization mechanisms such as RMSNorm and Query-Key Normalization, making parts of the model app