Training Trajectories Determine Circuit Removability in Annealable Soft-Prior Transformers

arXiv:2609.10287v1 Announce Type: new Abstract: Soft positional priors can help small Transformers learn retrieval circuits, but it is unclear whether the resulting circuits remain functional once the prior is removed. We test this with an annealable soft-prior Transformer whose attention biases can be learned, faded, or zeroed during training and evaluation. On associative recall, unforced models perform well with the prior active (0.772 pm 0.020) but collapse at zero gate (0.095 pm 0.009). Smooth fade-to-zero training preserves high zero-gate accuracy (0.734 pm 0.028), whereas forced-zero training, hard switching, and post hoc continuation fail to recover the same effect. The pattern also appears on Markov induction. Linear regression ICL provides a boundary case because zero-gate training can learn that task directly. Mechanistic traces show that circuit consolidation occurs after the gate reaches zero, even though the responsible heads vary across seeds. These results suggest that circuit removability in small discrete retrieval tasks depends on the training trajectory, not just the final architecture.

Related

• Transformers Provably Learn to Internalize Chain-of-Thought
• How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability
• Grounding Functional Similarity by Invariance-Aware Model Stitching
• Transformers with RL or SFT Provably Learn Sparse Boolean Functions, But Differently

Source: arXiv cs.LG | 2026-09-10