Research
CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge
arXiv:2604.14644v1 Announce Type: new Abstract: The inability to filter out in advance all potentially problematic data from the pre-training of large language models has given rise to the need for me
arXiv:2604.14644v1 Announce Type: new Abstract: The inability to filter out in advance all potentially problematic data from the pre-training of large language models has given rise to the need for methods for unlearning specific pieces of knowledge after training. Existing techniques overlook the need for continuous and immediate action, causing them to suffer from degraded utility as updates accumulate and protracted exposure of sensitive information. To address these issues, we propose Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge (CURaTE). Our method begins by training a sentence embedding model on a dataset designed to enable the formation of sharp decision boundaries for determining whether a given input prompt corresponds to any stored forget requests. The similarity of a given input to the forget requests is then used to determine whether to answer or return a refusal response. We show that even with such a simple approach, not only does CURaTE achieve more effective forgetting than existing methods, but by avoiding modification of the language model parameters, it also maintains near perfect knowledge preservation over any number of updates and is the only method capable of continual unlearning in real-time.
Related
- Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts
- Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
- Guaranteeing Knowledge Integration with Joint Decoding for Retrieval-Augmented Generation
- Self-Correcting RAG: Enhancing Faithfulness via MMKP Context Selection and NLI-Guided MCTS
Source: arXiv cs.CL | 2026-04-17