Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers
DGX agentarXiv:2608.07436v1 Announce Type: new Abstract: Under the standard split, Muon gets hidden matrices and AdamW embeddings/output head. Muon groks modular addition faster, but its solutions do not hold.