Beyond Top Words: MonoTM for Topic Modeling with Interpretable Monosemantic Features

arXiv:2609.09575v1 Announce Type: new Abstract: Topic models summarize large text corpora, but top-ranked words often provide only a limited representation of topic semantics. Sparse autoencoders (SAEs) offer a way to move beyond word-level descriptors by extracting interpretable features from dense representations, yet how feature interpretability relates to topic-inference quality remains unclear. We introduce extbf{MonoTM}, an interpretable topic modeling framework that decouples these roles. Across three benchmark corpora, we show that document--topic mixture estimation and semantic interpretation favor different SAE configurations and feature subsets. MonoTM estimates mixtures from the full SAE bag-of-features representation and, with them fixed, learns topic descriptors over a separate vocabulary of corpus-grounded semantic features. This design preserves global topic structure while representing topics with semantic units more meaningful than individual words, making them more useful for downstream corpus analysis.

Related

• CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features
• Control Reinforcement Learning: Interpretable Token-Level Steering of LLMs via Sparse Autoencoder Features
• Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders
• Feature Rivalry in Sparse Autoencoder Representations: A Mechanistic Study of Uncertainty-Driven Feature Competition in LLMs
• Domain-Agnostic Neural Topic Modeling with Contextual Token-Level Semantic Graph Representation

Source: arXiv cs.CL | 2026-09-10