VFA: Empowering Multilingual MLLMs via Vision-Free Adaptation

arXiv:2608.26155v1 Announce Type: cross Abstract: Multimodal large language models have advanced rapidly, yet most remain English-centric, as scaling multilingual multimodal instruction tuning is limited by the scarcity and high cost of high-quality non-English image-text supervision. Although multilingual text data is abundant, naive textual fine-tuning can disrupt vision-language alignment and induce catastrophic forgetting. We propose Vision-Free Adaptation (VFA), a framework that decouples multilingual language enhancement from visual alignment by composing complementary task vectors over a shared LLM backbone. Specifically, we fine-tune a base LLM on multilingual text data to derive a multilingual task vector, which is then merged with the vision-aligned task vector of an MLLM. Experiments on five MLLMs across six multilingual multimodal benchmarks show consistent improvements while preserving both general multimodal and text-only capabilities. Moreover, using less than 2% of the text data, VFA narrows the gap to the fully multimodal-trained model, demonstrating its data efficiency.

Related

• MELLA: Bridging Linguistic Capability and Cultural Groundedness for Low-Resource Language MLLMs
• Adapter Merging Reactivates Latent Reasoning Traces: A Mechanism Analysis
• ADAPT: Attention Dynamics Alignment with Preference Tuning for Faithful MLLMs

Source: arXiv cs.AI | 2026-08-28