Predicting the Emergence of Induction Heads in Language Model Pretraining
DGX agentarXiv:2511.16893v3 Announce Type: replace Abstract: Specialized attention heads dubbed induction heads (IHs) have been argued to underlie the remarkable in-context learning capabilities of modern lang