Approximate Homomorphisms and Convergent Representations in Transducers

arXiv:2608.20428v1 Announce Type: cross Abstract: We study the stability of minimal representations of controlled stochastic processes (in particular, transducers) under perturbations. This question is motivated by recent experiments finding predictive-state structure in the latent representations of neural networks. We consider standard, linear and predictive transducers. We introduce notions of approximate homomorphism capturing local structural similarity between them, together with metrics comparing their induced dynamics (which we refer to as interfaces), and prove properties such as composability of the approximate homomorphisms. For standard transducers, we show that there exist simple interfaces for which there is no approximate homomorphism between the different implementations of the dynamics. In contrast, for every finite-rank interface mathcal I, we prove that all minimal linear transducers implementing interfaces sufficiently close to mathcal I have an approximate homomorphism to the minimal implementation of mathcal I, with error linear in the perturbation size. We prove an analogous stability result for predictive transducers under a residual metric using some mild hypothesis regarding the indistinguishability of the belief states. These results identify conditions under which canonical transducer representations are robust to perturbations, while showing that such convergence fails without additional structural restrictions. Under the assumption that these type of abstractions are embedded into the hidden layers of modern AI models, this gives some theoretical support to the hypothesis that their latent representations exhibit structural convergence.

Related

• Two-Fidelity Best-Action Identification for Stochastic Minimax Tree
• Spectral Perturbation of the Empirical Fisher Information Matrix under Weight Quantization
• Learning to Hand Off: Provably Convergent Workflow Learning under Interface Constraints
• A Graph Signal Processing Perspective on Numerical Sequence Representations in LLM In-Context Learning

Source: arXiv cs.AI | 2026-08-24