EXAM^2: nderline{Ex}tending nderline{A}udio Understanding in nderline{M}ultilingual and nderline{M}ultimodal Analysis

arXiv:2608.23758v1 Announce Type: cross Abstract: Recent large audio language models (LALMs) have achieved impressive progress in audio understanding. However, existing evaluations remain largely constrained to English and narrow audio domains. Prior benchmarks typically focus on a single audio modality, i.e., speech, sound, or music, limiting the systematic investigation into how these models generalize across diverse visual scenarios. In this paper, we introduce EXAM^2, a benchmark for multilingual and multimodal audio understanding spanning six languages and multiple modalities, including speech, sound, music, mixed-audio settings, and visual images. By incorporating visual information alongside heterogeneous audio inputs, EXAM^2 enables more realistic evaluation of scene-aware audio reasoning and cross-modal comprehension. EXAM^2 comprises 5,667 multiple-choice questions, 22,614 image instances, and 135,684 multilingual translations. We evaluate state-of-the-art open-source and proprietary LALMs as well as multimodal LLMs, revealing substantial performance gaps in multilingual and cross-modal understanding. Furthermore, we propose Gemma3n-EXAM^2, a lightweight fusion-model fine-tuned on EXAM^2-train, achieves up to 12.4% improvement in multilingual settings and 21.7% gains in multimodal evaluation over a strong baseline. Empirical results establish EXAM^2 as a challenging benchmark and pioneer future multilingual and multimodal audio intelligence research.

Related

• KoALa-Bench: Evaluating Large Audio Language Models on Korean Speech Understanding and Faithfulness
• From Sounds to Scenes: A Benchmark for Evaluating Context-Aware Auditory Scene Understanding in Large Audio Language Models
• HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models
• AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs

Source: arXiv cs.AI | 2026-08-26