Donors and Recipients: On Asymmetric Transfer Across Tasks and Languages with Parameter-Efficient Fine-Tuning

arXiv:2511.13368v3 Announce Type: replace-cross Abstract: Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and languages remains poorly understood. We conduct a controlled LoRA fine-tuning study across multiple open-weight LLM families and scales, using a standardised grid of 11 languages and four benchmarks. We fine-tune each model on a single task-language source, then evaluate it on all other task-language target pairs to measure transfer. We decompose transfer into three regimes: (i) Matched-Task (Cross-Language), (ii) Cross-Task (Matched-Language), and (iii) Cross-Task (Cross-Language). Single-source fine-tuning yields a net positive uplift across regimes, but the gains are strongly asymmetric. Matched-Task (Cross-Language) transfer emerges as the most effective and structurally regular regime, with transfer magnitude driven principally by the identity of the target language rather than model architecture. We identify a stable coarse-grained hierarchy in which some task and language targets consistently absorb gains from diverse sources, while others remain relatively isolated. These results imply that effective fine-tuning requires accounting for donor-recipient roles to maximise downstream gains while limiting collateral degradation in other capabilities.

Related

• From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning
• PEML: Parameter-efficient Multi-Task Learning with Optimized Continuous Prompts
• Cross-Lingual Transfer and Parameter-Efficient Adaptation in the Turkic Language Family: A Theoretical Framework for Low-Resource Language Models
• COMPASS: COntinual Multilingual PEFT with Adaptive Semantic Sampling

Source: arXiv cs.AI | 2026-09-15