Research
Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder
arXiv:2506.20083v4 Announce Type: replace Abstract: Integrating compositional and symbolic properties into current distributional semantic spaces can enhance the interpretability, controllability, com
arXiv:2506.20083v4 Announce Type: replace Abstract: Integrating compositional and symbolic properties into current distributional semantic spaces can enhance the interpretability, controllability, compositionality, and generalisation capabilities of Transformer-based auto-regressive language models (LMs). In this survey, we offer a novel perspective on latent space geometry through the lens of compositional semantics, a direction we refer to as extit{semantic representation learning}. This direction enables a bridge between symbolic and distributional semantics, helping to mitigate the gap between them. We review and compare three mainstream autoencoder architectures-Variational AutoEncoder (VAE), Vector Quantised VAE (VQVAE), and Sparse AutoEncoder (SAE)-and examine the distinctive latent geometries they induce in relation to semantic structure and interpretability.
Related
- Current LLMs still cannot 'talk much' about grammar modules: Evidence from syntax
- Semantic-Space Exploration and Exploitation in RLVR for LLM Reasoning
- KCS: Diversify Multi-hop Question Generation with Knowledge Composition Sampling
- Discourse Coherence and Response-Guided Context Rewriting for Multi-Party Dialogue Generation
Source: arXiv cs.CL | 2026-04-16