Distribution-aware Language Neuron Identification in Multilingual Large Language Models

arXiv:2609.10993v1 Announce Type: new Abstract: Multilingual large language models (mLLMs) contain a small fraction of feed-forward neurons that are sensitive to particular languages, commonly termed language-specific neurons. Existing work measures language specificity using the entropy of each neuron's language-wise probabilities of being active, where a neuron is considered active when its activation value is positive. However, this approach may not fully capture the multilingual nature of mLLMs, where language representations are distributional and mutually related. We propose Distribution-aware Language Neuron selection, which leverages pairwise relationships between per-language activation distributions over the full activation range, including negative values. Specifically, we quantify each neuron's language specificity by clustering languages using pairwise overlap coefficients between their activation distributions. Across two mLLMs and two held-out corpora, our identifier more effectively isolates language-specific causal effects, yielding up to 4.9imes higher on-target language damage per neuron while preserving off-target language performance.

Related

• Multilingual Emotion Neurons in Large Audio-Language Models
• Are Emotion and Rhetoric Neurons in LLM? Neuron Recognition and Adaptive Masking for Emotion-Rhetoric Prediction Steering
• Shared Circuits for Shared Grammar: Tracing Subject-Verb Agreement Across Languages
• MLLMs Hallucinate when Information Distribution Drifts in Synergy Heads

Source: arXiv cs.CL | 2026-09-11