Research
SPICE: Simple Polysemantic Feature Interpretation via Clustering-based Explanation
arXiv:2609.13198v1 Announce Type: cross Abstract: One of the pivotal recent challenges in neural network interpretability is polysemanticity, where a single neuron is activated by multiple, often unre
arXiv:2609.13198v1 Announce Type: cross Abstract: One of the pivotal recent challenges in neural network interpretability is polysemanticity, where a single neuron is activated by multiple, often unrelated concepts, hindering clear functional understanding. Although prior work has explored this phenomenon, existing approaches remain architecture-specific and depend on manual heuristics such as a fixed number of concept clusters (K), limiting their generality and scalability--especially for modern Transformer-based models. To address these limitations, we introduce SPICE (extbf{S}imple extbf{P}olysemantic Feature extbf{I}nterpretation via extbf{C}lustering-based extbf{E}xplanation), a generalizable framework for analyzing polysemanticity in deep vision architectures. SPICE avoids architecture-dependent propagation rules, enabling the first systematic comparison of polysemanticity across both CNNs and Transformers, and automatically determines the number of concept clusters per neuron, eliminating reliance on a preset K and supporting scalable analysis for large models. Using SPICE, we conduct a comprehensive investigation into how polysemanticity emerges, varies across depth and architecture, and forms through distinct computational pathways.
Related
- SaliencyDecor: Enhancing Neural Network Interpretability through Feature Decorrelation
- Observable- and Positional-Encoding-Dependent Symmetry Readout from Neural Network Weights
- ICON Decomposition: Multivariate Concept-Level Explanations of Deep Representations for Model Auditing
Source: arXiv cs.CV | 2026-09-15