LayerNorm Induces Recency Bias in Transformer Decoders
DGX agentarXiv:2509.21042v3 Announce Type: replace Abstract: Causal self-attention provides positional information to Transformer decoders. Prior work has shown that stacks of causal self-attention layers alon