We need a better taxonomy for what people are calling 'continual learning'

Continual learning isn't some fake term but a real goal and arguably one of the most important open ones in 2026. It roughly means: ‘building systems that keep acquiring useful knowledge or skills after deployment without a full retrain’. IMO, the problem is that it's a problem setting being used as if it named a mechanism. When I talk to someone who says "we do continual learning," I now have to guess whether they mean gradients running in prod, a markdown file the agent appends to, a 5M-token context or a nightly distillation job. Here's how I'd break it down: long-context ICL, text optimization, recurrent latent memory, per-task test-time training, and online parametric continual fine-tuning 1. long-context ICL Weights stay frozen and the model learns the task from context optimization, i.e., by conditioning on demos, feedback, and history in the current context. The idea is that a sufficiently large context window could provide enough ICL for an agent to learn a task without fine-tuning. Key limitation: a very large working memory and no transfer from current context to long-term memory. 2. text optimization over the mutable text layer The system rewrites the mutable text layer around a frozen model: system prompts, skill files, playbooks, memory stores, retrieval indices, harness code. Important tradeoff: forgetting doesn't vanish here, it moves from weight interference to memory construction and retrieval, where old and new experiences still compete for a bounded context. 3. recurrent/architectural latent memory with frozen weights Task info gets written into an evolving internal state not a growing KV cache or some text file and parameters stay fixed. Multi-timescale self-modifying architectures sit here. BDH-CQ is another clean illustration of the pattern: each demonstration from the train-test set is integrated into a recurrent memory that gradually builds an internal representation of the task. This is the starting point for reasoning about test-test inputs in a separate reasoning loop. The model thus adapts at inference time through state updates alone, without modifying its parameters 4. per-task test-time training with gradients The system turns demonstrations into a small training set, performs gradient updates for the current task, generates an answer, and may then discard those updates. Some ARC pipelines (including the evaluated HRM/TRM setups) use this kind of task-specific optimization. I’m not sure it is truly “continual” if nothing persists across tasks, but it frequently gets grouped under that label. 5. online parametric continual fine-tuning Gradients are applied persistently after deployment. The main challenges are finding good labels/rewards at test time and learning new information without destroying existing capabilities, using techniques such as replay, regularization, parameter isolation, or sparse and targeted updates. Some self-editing systems are hybrids: the model produces its own fine-tuning data in text, but the resulting update is stored in weights. Imp tradeoff: catastrophic forgetting and accumulated weight updates may not survive a base-model upgrade. It’s also hard to find proper signals from which we can back-propagate at inference time. That’s how I have split the term and I’m curious where people here disagree and if I missed any. submitted by /u/Typical-Scene-5794 [link] [comments]

Source: r/LocalLLaMA | 2026-09-03