Research
CircuitTree: Composing Circuits into A Syllable Transcriber
arXiv:2610.12520v1 Announce Type: cross Abstract: The goal of circuit discovery is to decompose a monolithic neural network into tiny modular subnetworks which are interpretable in the sense that each
arXiv:2610.12520v1 Announce Type: cross Abstract: The goal of circuit discovery is to decompose a monolithic neural network into tiny modular subnetworks which are interpretable in the sense that each does an identifiable subtask. We develop a practical method for composing arbitrarily many circuits into a single model. We propose a greedy, density-scaled composition algorithm that overcomes limitations of previous attempts to compose circuits by naive union, then generalize it to N circuits via CircuitTree, a new data structure built on a Theta(Nlog N) tree-compose schedule. We compose circuits pruned from HuBERT for classifying individual phonemes into a syllable classifier, a capability none of the 39 phoneme circuits has individually. Chaining CircuitTrees into a hierarchy-of-circuits pipeline, we transcribe open syllables at 64.8% accuracy, within 1.2 points of a 66.0% ceiling that shares no circuitry at all, with 99.5% of circuit components shared.
Related
- Individual Parameters in Weight-Sparse Transformers Appear Interpretable
- Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning
- Mixture of Layers with Hybrid Attention
- PathMoG: A Pathway-Centric Modular Graph Neural Network for Multi-Omics Survival Prediction
Source: arXiv cs.AI | 2026-10-12