The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

arXiv:2609.10296v1 Announce Type: new Abstract: Non-invasive speech decoding remains constrained by the low signal-to-noise ratio of neural recordings, which makes fine-grained reconstruction of phonemes or individual words difficult. Motivated by neuroscientific evidence that high-level semantic representations are distributed across cortical regions and evolve over slower temporal scales, we hypothesize that semantic content may provide a more suitable target for non-invasive decoding than low-level acoustic or lexical features. We introduce Brain2Semantics2Text, a method that reconstructs text through an intermediate semantic embedding space. Our model maps sentence-level MEG responses into a semantic manifold and then inverts the predicted embeddings into natural language. This semantic bottleneck enables recovery of high-level meaning without word-level alignment. We describe the core principles of the approach, its implementation, and the strategies used to mitigate the challenges of learning a reliable neural-to-semantic mapping. Finally, we compare against prior non-invasive Brain2Text methods and show improved sentence-level results.

Related

• Beyond Parallel Tracking: Interactive Multi-Feature Fusion Drives Semantic Reconstruction from Non-invasive Brain Recordings
• Brain-CLIPLM: Decoding Compressed Semantic Representations in EEG for Language Reconstruction
• Margin-Regularized Structured Semantic Alignment for Brain-Language Correspondence
• Cross-Subject Semantic Decoding with Shared-Space Alignment for Generalized Neural Representation Learning

Source: arXiv cs.CL | 2026-09-10