Approximating f-Divergences with Rank Statistics

arXiv:2601.22784v2 Announce Type: replace-cross Abstract: We introduce a rank-statistic approximation of f-divergences that avoids explicit density-ratio estimation by working directly with the distribution of ranks. For a resolution parameter K, we map the mismatch between two univariate distributions mu and nu to a rank histogram on { 0, ldots, K} and measure its deviation from uniformity via a discrete f-divergence, yielding a rank-statistic divergence estimator. We prove that the resulting estimator of the divergence is monotone in K, is always a lower bound of the true f-divergence, and we establish quantitative convergence rates for Koinfty under mild regularity of the quantile-domain density ratio. To handle high-dimensional data, we define the sliced rank-statistic f-divergence by averaging the univariate construction over random projections, and we provide convergence results for the sliced limit as well. We also derive finite-sample deviation bounds along with asymptotic normality results for the estimator. Finally, we empirically validate the approach by benchmarking against neural baselines and illustrating its use as a learning objective in generative modeling experiments.

Source: arXiv cs.LG | 2026-06-02