Unifying Detection and Adaptation in Task-Free Continual Learning

arXiv:2608.27070v1 Announce Type: cross Abstract: To mitigate catastrophic forgetting in downstream continual learning (CL) for large language models (LLMs), existing methods typically constrain parameter updates or introduce task-specific adaptation modules. However, these methods often rely on explicit task boundaries during training, limiting their applicability to realistic task-free scenarios. In this paper, we propose a extbf{Fi}sher-guided extbf{uni}fied (extbf{FiUni}) framework for batch-level task detection and parameter-efficient continual adaptation. FiUni is motivated by a key observation about the Fisher information matrix (FIM) of pre-trained models: the orthogonality among the principal subspaces of its Kronecker-Factored Approximate Curvature (K-FAC) approximation, estimated from a small number of downstream task samples, can reflect the similarity between different tasks. Based on this observation, FiUni constructs FIM-derived frozen subspaces to guide low-rank adaptation (LoRA), while matching the Fisher principal subspace of each incoming batch window with historical subspaces. This enables FiUni to adaptively determine whether to reuse existing knowledge, expand a related subspace, or create a new subspace, dynamically balancing knowledge sharing and task isolation. Experiments show that FiUni can effectively infer latent batch-level task affiliations and achieve competitive performance against advanced task-aware CL methods with fewer trainable parameters.

Related

• CP-MoE: Consistency-Preserving Mixture-of-Experts for Continual Learning
• FOREVER: Forgetting Curve-Inspired Memory Replay for Language Model Continual Learning
• Parameter Alignment Mitigates Catastrophic Forgetting in Multilingual Expert Language Models
• Parameter Importance is Not Static: Evolving Parameter Isolation for Supervised Fine-Tuning
• Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updates

Source: arXiv cs.CL | 2026-08-28