When a neural network learns a new task, it typically adjusts its weights to minimize error on new data. However, in a monolithic architecture, these weights are the same ones used to solve previous tasks. This leads to catastrophic forgetting: the new gradients overwrite the old knowledge, leaving the model unable to perform its original duties.
To solve this, a growing body of research suggests moving away from monolithic models toward modular architectures. Instead of forcing one set of parameters to learn everything, these systems treat knowledge as a library of specialized "experts." By isolating new information into discrete units, we can achieve lifelong learning where the model grows in capacity without losing its past.
The Stability Plasticity Dilemma
At the heart of continual learning is the stability-plasticity dilemma. A model must be plastic enough to integrate new information but stable enough to prevent the degradation of existing skills.
Traditional approaches try to balance this through regularization—essentially telling the model "don't change these important weights too much." But as Mathieu Pagé Fortin notes in Class-Incremental Learning of Plant and Disease Detection, architectural expansion provides a "physical buffer." By adding new branches or modules for new tasks, the system ensures that the parameters responsible for old knowledge remain literally untouched.
Expanding the Architecture
The most direct way to implement this is through Dynamic Architectural Expansion. Instead of a fixed-size network, the model grows as it encounters new data distributions.
One of the foundational patterns for this is the Progressive Neural Network, introduced by Andrei A. Rusu and Razvan Pascanu. In Progressive Neural Networks, the system instantiates a new neural "column" for every new task. While the new column can "sideways-reference" features from previous columns to speed up learning, the original columns are frozen. This makes the model immune to forgetting by design.
Similar logic is applied in reinforcement learning. Tiantian Zhang and Deheng Ye developed DaCoRL (Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization), which uses Bayesian clustering to detect when an environment has changed. Once a shift is detected, the model automatically adds a specialized output head to handle the new context.
Routing to the Right Expert
If a system has dozens or hundreds of specialized modules, how does it know which one to use for a given input? This is the routing problem.
Researchers have proposed several mechanisms to navigate this "library of experts":
- Gating Autoencoders: Rahaf Aljundi and colleagues proposed Expert Gate (Expert Gate: Lifelong Learning with a Network of Experts), which uses lightweight autoencoders to "screen" incoming data. Each expert has a corresponding gate that measures how well it recognizes the input; the data is then routed to the expert with the highest familiarity.
- Retrieval-of-Experts (RoE): In the realm of Large Language Models, Joel Jang and Seungone Kim demonstrated in Exploring the Benefits of Training Expert Language Models over Instruction Tuning that a library of small, task-specific models can outperform a single massive multitask model. They use a retrieval mechanism to pick the best expert for a prompt, allowing for "zero-cost" continual learning—adding a new skill is as simple as adding a new file to a folder.
- Semantic Routers: For question answering, Kyungjae Lee and Joonsuk Park proposed a "plug-and-play" architecture in Plug-and-Play Adaptation for Continuously-updated QA. They use an "Adaptation Switch" to trigger specific adapter modules based on the semantic content of the query.
Efficiency through Parameter Isolation
A common critique of modular growth is that it could lead to an explosion in model size. Recent work addresses this by using Parameter-Efficient Fine-Tuning (PEFT). Instead of adding entire neural columns, researchers add tiny "adapters" or "prompts."
For example, Hyunji Lee and Kyle Lo introduced RouterRetriever (RouterRetriever: Exploring the Benefits of Routing over Multiple Expert Embedding Models), which uses LoRA (Low-Rank Adaptation) modules as domain-specific experts. This allows the system to support many specialized domains with minimal memory overhead. Similarly, Minqian Liu and Lifu Huang use Episodic Memory Prompts (EMP) (Incremental Prompting: Episodic Memory Prompt for Lifelong Event Detection), where the model grows by appending new learnable continuous prompts for each task, creating a "soft memory" that preserves old knowledge.
Why It Works: The Power of Composition
The ultimate benefit of modularity isn't just preventing forgetting; it's enabling composition. As Andrea Cossu argues in Task-Agnostic Experts Composition for Continual Learning, when we break a system into specialized, frozen components, we can ensemble them in novel ways to solve tasks the model has never seen before.
By treating deep learning not as the training of a single monolithic "brain," but as the assembly of a "council of experts" (as described by Minghan Li 0002 in Multi-Task Dense Retrieval via Model Uncertainty Fusion), we create AI that is more scalable, more interpretable, and—most importantly—capable of learning forever.