Research
eta-VAEs as Effective Theories: Tolerance-Dependent Dimension
arXiv:2608.10599v1 Announce Type: new Abstract: In a eta-VAE, increasing the regularization strength acts as a spectral cutoff by collapsing low-utility latent coordinates. In the linear Gaussian VAE,
arXiv:2608.10599v1 Announce Type: new Abstract: In a eta-VAE, increasing the regularization strength acts as a spectral cutoff by collapsing low-utility latent coordinates. In the linear Gaussian VAE, the collapse order matches the ranking of reconstruction utilities exactly, because both are set by the PCA spectrum. We ask which parts of this picture survive in fully connected nonlinear VAEs trained on WorldClim. We find that nonlinear interactions shift and broaden collapse onsets, so thresholds no longer coincide exactly with utilities. However, the common ordering is preserved over the resolved ranks, so the spectral cutoff still acts as a utility cutoff and the effective-description logic carries through. The resulting effective-dimension curves reveal a head--tail tradeoff: increasing depth concentrates utility into the first few coordinates but worsens tail fidelity.
Related
- PDGMM-VAE: A Variational Autoencoder with Adaptive Per-Dimension Gaussian Mixture Model Priors for Nonlinear ICA
- PCAE: Learning Ordered Representations in Latent Space for Intrinsic Dimension Estimation via Principal Component Autoencoder
- Beyond Negative-Ridge Endpoints: Mixed-Sign Spectral Regularization via Negative-Shifted Gradient Descent
- Taming Audio VAEs via Target-KL Regularization
Source: arXiv cs.LG | 2026-08-12