Efficient Frechet Distributional Decoder Alignment for Latent Generative Models

arXiv:2610.17466v1 Announce Type: new Abstract: Frechet Distributional (FD) objectives provide an effective distribution-level signal for post-training generative models. For latent generative models, however, existing FD-based post-training methods optimize only the latent generator while keeping the decoder frozen: the decoder is trained on reconstruction latents obtained from real images but deployed on generator-produced latents at generation time. When these latent distributions differ, this extbf{decoder train-generation mismatch} means that the decoder is deployed on inputs drawn from a distribution it was not directly trained on. We introduce extbf{Frechet Distributional Decoder Alignment (FDDA)}, an efficient post-training method that freezes the latent generator and directly optimizes the decoder on generation-time latents using an FD objective. A single epoch of FDDA consistently improves gFID and gFDr^6 across diverse flow-based and autoregressive latent generative models. Across all 11 evaluated models, FDDA reduces gFDr^6 by 34.0-59.6% from the original pretrained models and provides a further 10.2-57.2% reduction after generator-side FD post-training. FDDA also reduces total wall-clock time by factors of 3.6imes-328.7imes relative to generator-side FD post-training across these models. These results show that generator-side post-training alone does not exhaust the benefits of FD optimization, and that further gains can be obtained by adapting the decoder at substantially lower cost.

Related

• AdvFD: Boosting Visual Generation via Adversarial Fr'echet Distance Loss
• Equivariant Latent Alignment via Flow Matching under Group Symmetries
• Oracle Noise: Faster Semantic Spherical Alignment for Interpretable Latent Optimization

Source: arXiv cs.CV | 2026-10-14