Attention is Just Another Name for Coupling? A Fast-Slow ODE Perspective on Hierarchical Pretraining
DGX agentarXiv:2606.16730v2 Announce Type: replace-cross Abstract: We re-interpret Transformer pretraining as a fast-slow, singularly perturbed flow along depth, with untied weights as its non-autonomous featu