Label Differential Privacy via Aggregation

arXiv:2310.10092v4 Announce Type: replace Abstract: This paper explores the use of linear aggregation to protect the privacy of sensitive training labels through the concept of label differential privacy (label-DP) while maintaining regression task utility. Our key finding is that weighted linear aggregation of training instances with i.i.d. N(0, 1) weights can achieve (arepsilon, elta)-label-DP with m = Oleft(n/(log(1/elta))right). Unlike prior methods, our approach relies on the minimum linear regression loss rather than the minimum singular value of the data matrix, resulting in better practical bounds on real datasets. We also examine real-world mechanisms involving disjoint sets or extit{bags} of instances. We demonstrate that aggregating labels from sub-sampled disjoint k-sized bags using i.i.d. N(0,1) weights achieves (arepsilon,elta)-label-DP with k geq Omegaleft(left((1/arepsilon)logleft(1/eltaright)right)^2right). In both scenarios, the optimal linear mse-regressor on the aggregated data approximates the original dataset's optimum with high probability, without needing additive label noise. Furthermore, we show that adding N(0,1) noise to any constant fraction of labels allows for similar label-DP guarantees when aggregating labels over random disjoint bags, while preserving the utility of Lipschitz-bounded neural mse-regression tasks.

Related

• Informationally Compressive Anonymization: Non-Degrading Sensitive Input Protection for Privacy-Preserving Supervised Machine Learning
• A Differentially Private Weighted Empirical Risk Minimization Procedure and its Application to Outcome Weighted Learning
• Server-Proximal Aggregation for Federated Domain-Incremental Learning under Partial Participation: Task-Uniform Convergence and Backward Transfer

Source: arXiv cs.LG | 2026-09-11