Near-Optimal Pure Machine Unlearning for Smooth Strongly Convex Losses

arXiv:2606.01527v1 Announce Type: new Abstract: Machine unlearning is motivated by legal and user-facing requirements to remove the influence of individuals' data from trained models, such as the right to be forgotten. Prior work has developed algorithms and error bounds for unlearning in smooth strongly convex stochastic optimization, but the fundamental statistical cost of unlearning has remained unclear. We nearly resolve this problem by proving upper and lower bounds on the excess population risk of approximate arepsilon-unlearning; our bounds are tight up to a condition-number factor. For mean estimation over the unit ball, our upper and lower bounds match. The optimal rate is the usual statistical error plus an unlearning penalty that interpolates between the retraining-from-scratch rate and an exponentially smaller term as arepsilon/d grows, where d is the dimension of the model. In particular, when arepsilon gg d, our arepsilon-unlearning algorithm offers an exponential accuracy improvement over retraining the model from scratch and differentially private baselines. On the other hand, when arepsilon le d, retraining from scratch is optimal.

Source: arXiv cs.LG | 2026-06-02