Subcritical Signal Propagation at Initialization in Normalization-Free Transformers
DGX agentarXiv:2604.11890v1 Announce Type: new Abstract: We study signal propagation at initialization in transformers through the averaged partial Jacobian norm (APJN), a measure of gradient amplification acr