Training Trajectories Determine Circuit Removability in Annealable Soft-Prior Transformers
arXiv:2609.10287v1 Announce Type: new Abstract: Soft positional priors can help small Transformers learn retrieval circuits, but it is unclear whether the resulting circuits remain functional once the