Lost and Found in Translation: Variational Diagnostics for Neural Codebook Channels

arXiv:2605.18846v1 Announce Type: cross Abstract: Classical communication systems fail not only through random noise but also when transmitter and receiver use incompatible operational codebooks. Variational autoencoders (VAEs) train an encoder q_phi and decoder p_heta jointly, and practitioners treat the resulting latent space as a discrete code -- for clustering, conditional generation, and mechanistic interpretability. Yet standard VAE diagnostics -- ELBO, active units, mutual information, and code histograms -- certify only whether this code is used, never whether the decoder reads each latent under the encoder's code. We close this gap with the neural codebook channel K_{eo d}(jmid i), a coupled encoder-decoder diagnostic whose off-diagonal mass is bounded by an architecture-free Bernoulli-KL certificate d_{bin}(1-A ,|, areta_p) le arDelta controlled by the variational gap. The certificate is the operational specialization of the classical KL chain rule under disintegration to the encoder-decoder disagreement event, complemented by a constructive marginal-impossibility result: no combination of marginal histograms, entropies, active-code counts, or mutual information determines K_{eo d}. We audit the certificate on four sklearn datasets (finite-grid exact, 5/5 seeds, 20/20 pairs satisfy the bound), a 2D model where the bound is non-vacuous at 2.71imes the observed disagreement and the four-term identity closes within 10^{-4}, MNIST under importance-sampling control, and a VQ-VAE attaining the predicted limit hat{A}=1.000. The package (K_{eo d}, A, R_{eff}, R, AU) is an audit-ready reporting unit. More broadly, the framework makes mismatched decoding -- a failure mode classical communication theory named decades ago -- visible inside a single deep generative model.

Source: arXiv cs.AI | 2026-05-20