Gradual Code-Switching as Inference-Time Cross-Lingual Representational Alignment for LLMs

arXiv:2510.05678v2 Announce Type: replace-cross Abstract: While large language models (LLMs) have achieved notable progress in multilingual settings, their performance remains uneven across languages as LLMs often rely on English-centric latent representations. In this work, we introduce code-switching in-context learning (CSICL), an inference-time mechanism for cross-lingual representational alignment. Rather than relying on an abrupt translation pivot, CSICL explicitly scaffolds the reasoning trajectory by gradually transitioning from a target language to English, aligning non-English inputs with an English-centric reasoning space. Across 4 LLMs, 6 datasets, and 10 languages, CSICL consistently outperforms cross-lingual in-context learning baselines, yielding average gains of 6.0pp and 4.8pp in target and unseen languages, respectively. The improvements generalize across language families and are even more pronounced in low-resource settings, with gains of 14.7pp in target and 5.3pp in unseen languages. These findings establish code-switching as a robust and effective approach for reducing cross-lingual misalignment during inference, moving LLMs towards more equitable and effective multilingual systems.

Related

• Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance
• Toward Robust Multilingual Adaptation of LLMs for Low-Resource Languages
• Sycophancy as a Multilingual Alignment Failure: How Safety Degrades Across Languages, Topics, and Models

Source: arXiv cs.AI | 2026-08-14