From Anchors to Supervision: Memory-Graph Guided Corpus-Free Unlearning for Large Language Models

arXiv:2604.13777v1 Announce Type: new Abstract: Large language models (LLMs) may memorize sensitive or copyrighted content, raising significant privacy and legal concerns. While machine unlearning has emerged as a potential remedy, prevailing paradigms rely on user-provided forget sets, making unlearning requests difficult to audit and exposing systems to secondary leakage and malicious abuse. We propose MAGE, a Memory-grAph Guided Erasure framework for user-minimized, corpus-free unlearning. Given only a lightweight user anchor that identifies a target entity, MAGE probes the target LLM to recover target-related memorization, organizes it into a weighted local memory graph, and synthesizes scoped supervision for unlearning. MAGE is model-agnostic, can be plugged into standard unlearning methods, and requires no access to the original training corpus. Experiments on two benchmarks, TOFU and RWKU, demonstrate that MAGE's self-generated supervision achieves effective unlearning performance comparable to supervision generated with external reference, while preserving overall utility. These results support a practical and auditable unlearning workflow driven by minimal anchors rather than user-supplied forget corpora.

Related

• RePAIR: Interactive Machine Unlearning through Prompt-Aware Model Repair
• Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models
• Psychological Concept Neurons: Can Neural Control Bias Probing and Shift Generation in LLMs?
• Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attribution
• Look Twice before You Leap: A Rational Framework for Localized Adversarial Anonymization

Source: arXiv cs.CL | 2026-04-16