Linguistic Distance Segregates Latent Representations in Automatic Speech Recognition Systems

arXiv:2608.30853v1 Announce Type: new Abstract: While automatic speech recognition (ASR) models have achieved remarkable improvements in recent years, performance disparities persist across different speaker populations. One such disparity is for speakers whose first languages (L1) are from families distant from English. This paper investigates the relationship between first language background and English ASR performance. Through empirical analysis, we observe that the correlation between speakers' L1 distance and ASR error rates yields a systematic effect on English Speech, with its strength varying across datasets and models. This association is statistically significant in a follow-up analysis accounting for dataset-level variation in Tweedie mixed-effects models (p<0.001 across evaluated models). In addition, analysis of the latent space reveals a L1-based spatial segregation across deeper acoustic layers in the majority of evaluated architectures

Related

• Fine-Tuning Whisper for Automatic Speech Recognition in Baniwa: A Preliminary Study
• Evaluating Japanese Dialect Robustness Across Speech and Text-based Large Language Models
• MELD: Mel-Spectrogram-Based Speech Language Modeling with Discrete Latent Variables

Source: arXiv cs.CL | 2026-09-01