Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis

arXiv:2506.00849v2 Announce Type: replace Abstract: Despite the empirical success of Diffusion Models (DMs) and Variational Autoencoders (VAEs), their generalization performance remains theoretically underexplored, especially lacking a full consideration of the shared encoder-generator structure. Leveraging recent information-theoretic tools, we propose a unified theoretical framework that provides guarantees for the generalization of both the encoder and generator by treating them as randomized mappings. This framework further enables (1) a refined analysis for VAEs, accounting for the generator's generalization, which was previously overlooked; (2) illustrating an explicit trade-off in generalization terms for DMs that depends on the diffusion time T; and (3) providing computable bounds for DMs based solely on the training data, allowing the selection of the optimal T and the integration of such bounds into the optimization process to improve model performance. Empirical results on both synthetic and real datasets illustrate the validity of the proposed theory.

Related

• From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models
• Generalization, memorization, and overfitting for diffusion models trained in the lazy high-dimensional regime
• An Analysis of Posterior Collapse, Parameterization and Initialization in Variational Deep Gaussian Processes
• Generalizing Score-based generative models for Heavy-tailed Distributions
• Generalization Properties of Score-matching Diffusion Models for Intrinsically Low-dimensional Data

Source: arXiv cs.LG | 2026-09-11