IMU: Influence-guided Machine Unlearning

arXiv:2508.01620v3 Announce Type: replace-cross Abstract: Machine Unlearning (MU) aims to selectively erase the influence of specific data points from pretrained models. However, most existing MU methods rely on the retain set to preserve model utility, which is often impractical due to privacy restrictions and storage constraints. While several retain-data-free methods attempt to bypass this using geometric feature shifts or auxiliary statistics, they typically treat forgetting samples uniformly, overlooking their heterogeneous contributions. To address this, we propose l{I}nfluence-guided l{M}achine l{U}nlearning (IMU), a principled method that conducts MU using only the forget set. Departing from uniform Gradient Ascent (GA) or implicit weighting mechanisms, IMU leverages influence functions as an explicit priority signal to allocate unlearning strength. To circumvent the prohibitive cost of full-model Hessian inversion, we introduce a theoretically grounded classifier-level influence approximation. This efficient design allows IMU to dynamically reweight unlearning updates, aggressively targeting samples that most strongly support the forgetting objective while minimizing unnecessary perturbation to retained knowledge. Extensive experiments across vision and language tasks show that IMU achieves highly competitive results. Compared to standard uniform GA, IMU maintains identical unlearning depth while enhancing model utility by an average of 30%, effectively overcoming the inherent utility-forgetting trade-off.

Related

• Efficient Unlearning through Maximizing Relearning Convergence Delay
• CoLA: A Choice Leakage Attack Framework to Expose Privacy Risks in Subset Training
• DMin: Scalable Training Data Influence Estimation for Diffusion Models
• GrOCE:Graph-Guided Online Concept Erasure for Text-to-Image Diffusion Models

Source: arXiv cs.CV | 2026-04-15