Optimal Stability of KL Divergence under Gaussian Perturbations

arXiv:2604.11026v1 Announce Type: cross Abstract: We study the problem of characterizing the stability of Kullback-Leibler (KL) divergence under Gaussian perturbations beyond Gaussian families. Existing relaxed triangle inequalities for KL divergence critically rely on the assumption that all involved distributions are Gaussian, which limits their applicability in modern applications such as out-of-distribution (OOD) detection with flow-based generative models. In this paper, we remove this restriction by establishing a sharp stability bound between an arbitrary distribution and Gaussian families under mild moment conditions. Specifically, let P be a distribution with finite second moment, and let N_1 and N_2 be multivariate Gaussian distributions. We show that if KL(P||N_1) is large and KL(N_1||N_2) is at most epsilon, then KL(P||N_2) ge KL(P||N_1) - O(sqrt{epsilon}). Moreover, we prove that this sqrt{epsilon} rate is optimal in general, even within the Gaussian family. This result reveals an intrinsic stability property of KL divergence under Gaussian perturbations, extending classical Gaussian-only relaxed triangle inequalities to general distributions. The result is non-trivial due to the asymmetry of KL divergence and the absence of a triangle inequality in general probability spaces. As an application, we provide a rigorous foundation for KL-based OOD analysis in flow-based models, removing strong Gaussian assumptions used in prior work. More broadly, our result enables KL-based reasoning in non-Gaussian settings arising in deep learning and reinforcement learning.

Related

• Audio Source Separation in Reverberant Environments using eta-divergence based Nonnegative Factorization

Source: arXiv cs.AI | 2026-04-14