Transforming Rank: How Architecture Navigates the Spectral Pathologies of Depth
arXiv:2607.14018v1 Announce Type: cross Abstract: We investigate how each component of the Transformer feedforward block architecture design determines how much rank survives across depth at initializ