Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation

arXiv:2604.19678v1 Announce Type: new Abstract: Function vectors (FVs) are vector representations of tasks extracted from model activations during in-context learning. While prior work has shown that multilingual model representations can be language-agnostic, it remains unclear whether the same holds for function vectors. We study whether FVs exhibit language-agnosticity, using machine translation as a case study. Across three decoder-only multilingual LLMs, we find that translation FVs extracted from a single EnglishrightarrowTarget direction transfer to other target languages, consistently improving the rank of correct translation tokens across multiple unseen languages. Ablation results show that removing the FV degrades translation across languages with limited impact on unrelated tasks. We further show that base-model FVs transfer to instruction-tuned variants and partially generalize from word-level to sentence-level translation.

Related

• On Temperature-Constrained Non-Deterministic Machine Translation: Potential and Evaluation
• Chinese Essay Rhetoric Recognition Using LoRA, In-context Learning and Model Ensemble
• How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models
• Graph In-Context Operator Networks for Generalizable Spatiotemporal Prediction

Source: arXiv cs.CL | 2026-04-22