Hearing the Order: Investigating Position Bias in Large Audio-Language Models

arXiv:2510.00628v3 Announce Type: replace-cross Abstract: Large audio-language models (LALMs) are often used in tasks that involve reasoning over ordered options. An open question is whether their predictions are influenced by the order of answer choices, which would indicate a form of position bias and undermine their reliability. In this paper, we identify and analyze this problem in LALMs. We demonstrate that no model is immune to this bias through extensive experiments on six LALMs across three widely used benchmarks and their spoken counterparts. Shuffling the order of answer options can cause performance fluctuations of up to 24% and even change model rankings, raising concerns about the reliability of current evaluation practices. We also study permutation-based strategies and show that they can mitigate bias in most cases. Our work represents the first systematic investigation of this issue in LALMs, and we hope it raises awareness and motivates further research in this direction.

Related

• VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech
• Ranked by Position: Order Sensitivity as an Exploitable Attack Surface in LLM Listwise Recommenders
• Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

Source: arXiv cs.CL | 2026-09-01