Research
Causal Interventions Reveal Typologically Organized Syntactic Mechanisms in Multilingual Language Models
arXiv:2608.28924v1 Announce Type: new Abstract: Linguistic theory has long recognized cross-linguistic syntactic regularities, leading to claims that these similar structures are processed by similar
arXiv:2608.28924v1 Announce Type: new Abstract: Linguistic theory has long recognized cross-linguistic syntactic regularities, leading to claims that these similar structures are processed by similar mechanisms. However, this hypothesis has been difficult to test empirically due to our lack of fine-grained, manipulable access of human processing mechanisms. In this work, we take advantage of techniques from mechanistic interpretability to study such a question in multilingual LMs. We first isolate language-internal mechanisms before attempting to transfer them cross-lingually. Across four models and three well-studied constructions (subject--verb number agreement, anaphoric pronoun gender agreement, and filler--gap object extraction) we find consistent cross-lingual mechanism transfer. We further find transfer to be graded, with more transfer between more typologically similar languages. We believe our work provides novel hypotheses about cross-linguistic syntactic structures and multilingual processing, and more broadly shows how the study of language models can help inform linguistic theory.
Related
- Causal Drawbridges: Characterizing Gradient Blocking of Syntactic Islands in Transformer LMs
- Shared Circuits for Shared Grammar: Tracing Subject-Verb Agreement Across Languages
- Evaluating Dedicated Monolingual and Joint Multilingual Causal Models for Dravidian Languages
Source: arXiv cs.CL | 2026-09-01