Gated Normalization Removal and Scale Anchoring in Pre-Norm Transformers
arXiv:2602.10408v2 Announce Type: replace-cross Abstract: Normalization layers are standard in transformers, but it is not clear whether their sample-dependent computations are necessary throughout bo