Differential Privacy of Gradient Descent on Perturbed Objectives

arXiv:2610.02716v1 Announce Type: new Abstract: Objective perturbation adds a random linear term to a regularized empirical risk and releases the exact perturbed minimizer. We study the finite computation obtained by releasing the N-th iterate of deterministic gradient descent on wmapsto F(w;S)+langle z,wrangle, where zsimmathcal N(0,sigma^2I_d) is drawn once before optimization. For strongly convex and smooth objectives with Lipschitz Hessian, we prove an explicit condition under which the map zmapsto w_N is a C^1-diffeomorphism on the bounded domains used in the privacy argument, with a quantitative lower bound on the smallest singular value of its Jacobian. This permits a direct change-of-variables analysis of the finite iterate. For generalized linear models, the resulting privacy-profile bound has no explicit ambient-dimension factor once the iteration condition holds, and its finite-iteration correction decreases geometrically. By letting the free truncation parameter grow slowly with N, we recover the corresponding exact-minimizer certificate in the limit. We also bound the expected excess empirical risk by dsigma^2/(2mu) plus a geometrically decreasing optimization term, and transfer the result to population risk without an additional multiplicative condition-number factor in the leading statistical terms.

Related

• Deep Learning under Fractional-Order Differential Privacy
• Trade-off Functions for DP-SGD with Subsampling based on Random Shuffling: Tight Upper and Lower Bounds
• Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?

Source: arXiv cs.LG | 2026-10-05