Can Renyi Accountants Beat Lossless Accountants for DP-SGD?

arXiv:2610.17120v1 Announce Type: new Abstract: Recent frameworks in differential privacy (DP), such as privacy loss distributions and f-DP, provide lossless representations of the privacy leakage of mechanisms. These approaches are often considered to yield tighter (arepsilon, elta)-DP guarantees than those obtained from relaxed notions such as Renyi DP. But is that always the case? We study this question for differentially private stochastic gradient descent (DP-SGD) with fixed-size batches under replace adjacency. Our key observation is that in this setting, privacy accounting based on Renyi DP can use a pair of worst-case distributions (i.e., a dominating pair) that differs from the one used in the lossless frameworks. In particular, we characterize the worst-case distributions as two-dimensional Gaussian mixtures and derive an improved Renyi DP bound for DP-SGD. Numerical and asymptotic comparisons show that across various hyperparameter settings, a Renyi DP accountant based on the new bound can yield better privacy guarantees than those obtained from the lossless frameworks.

Source: arXiv cs.LG | 2026-10-14