Transformer MLP Gate Thresholds Are Couplings to a Carried Reference Direction
arXiv:2609.31956v1 Announce Type: cross Abstract: The corpus-mean direction of a transformer's residual stream is a component shared across all inputs, and is commonly removed by mean-centering before