Realistic Continual Learning Approach using Pre-trained Models

arXiv:2404.07729v2 Announce Type: replace-cross Abstract: Continual learning (CL) evaluates adaptability in learning solutions to retain knowledge. Our research addresses the challenge of catastrophic forgetting, where models lose proficiency in previously learned tasks as they acquire new ones. While numerous solutions have been proposed, existing experimental setups often rely on idealized class-incremental learning scenarios. We introduce Realistic Continual Learning (RealCL), a novel CL paradigm where class distributions across tasks are random. We also present CLARE (Continual Learning Approach with pRE-trained models for RealCL scenarios), a pre-trained model-based solution designed to integrate new knowledge while preserving past learning. Our contributions include pioneering RealCL as a generalization of traditional CL setups, proposing CLARE as an adaptable approach for RealCL tasks, and conducting extensive experiments demonstrating its effectiveness across various RealCL scenarios. Notably, CLARE outperforms existing models on RealCL benchmarks, highlighting its versatility in unpredictable learning environments. Code to reproduce all our experiments can be found at https://github.com/gramuah/clare.

Related

• Continual Learning with Vision-Language Models via Semantic-Geometry Preservation
• Learning to Restore More: Continual Capability Expansion for Pretrained Image Restoration Models
• Little by Little: Continual Learning via Incremental Mixture of Rank-1 Associative Memory Experts

Source: arXiv cs.CV | 2026-09-15