Abstract
Continual learning is essential for real-world deployment when there is a need to quickly adapt the model to new tasks without forgetting knowledge of old tasks. Existing work on continual sequence generation either always reuses existing parameters to learn new tasks, which is vulnerable to catastrophic forgetting on dissimilar tasks, or blindly adds new parameters for every new task, which could prevent knowledge sharing between similar tasks. To get the best of both worlds, in this work, we propose continual sequence generation with adaptive compositional modules to adaptively add modules in transformer architectures and compose both old and new modules for new tasks. We also incorporate pseudo experience replay to facilitate knowledge transfer in those shared modules. Experiment results on various sequences of generation tasks show that our framework can adaptively add modules or reuse modules based on task similarity, outperforming state-of-the-art baselines in terms of both performance and parameter efficiency.
Introduction
In the real world, AI models are rarely static. To remain useful, they must learn a stream of different tasks over time—a process known as continual learning. However, neural networks are notoriously prone to catastrophic forgetting, where training on a new task (such as news summarization) causes the model to abruptly lose its ability to perform previously learned tasks (such as SQL query generation).
Current strategies to combat this generally fall into two categories: parameter sharing and parameter isolation. Parameter sharing encourages knowledge transfer but risks interference between dissimilar tasks. Conversely, parameter isolation (often using "adapters") prevents forgetting by adding new modules for every task, but this approach is parameter-inefficient and prevents the model from sharing useful patterns between similar domains. This research introduces a framework that achieves the "best of both worlds" by adaptively composing modules within a Transformer architecture.
The Adaptive Compositional Framework
The proposed method integrates a two-stage approach into the Transformer architecture (specifically GPT-2) to determine the optimal balance between parameter reuse and expansion.
1. The Decision Stage: Hidden State Mixing
Before full training begins, the model undergoes a "scouting" phase. For each layer, the model evaluates all existing adapter modules from previous tasks alongside one brand-new candidate module. Using a technique called Hidden State Mixing, the model calculates a weighted average of the outputs from these modules.
To ensure the model makes a definitive choice rather than a blurry average, the authors apply an Entropy Loss that forces the selection of a single "winner" for each layer. Additionally, a Reuse Prior is used during initialization to favor existing modules, ensuring that new parameters are only added if the existing knowledge base is truly insufficient for the new task.
2. The Training Stage: Pseudo Experience Replay
Once the architecture for the new task is finalized—creating a unique "path" of old and new modules through the network—the model begins full training. To prevent forgetting within the shared modules, the framework employs Pseudo Experience Replay. The model generates "fake" examples of previous tasks, which are then mixed with the new task data. This process keeps the shared parameters grounded in previous knowledge while they adapt to new information.
Experimental Evaluation
The researchers evaluated the framework across eight different task sequences, categorized into two scenarios:
- Similar Tasks: Natural Language Generation (NLG) across five domains (e.g., hotels, restaurants, laptops) using the E2ENLG and RNNLG datasets.
- Dissimilar Tasks: High-variance tasks including SQL query generation (WikiSQL), news summarization (CNN/DailyMail), and dialogue state tracking (MultiWOZ).
Key Results and Insights
- Superior Performance: On sequences of dissimilar tasks, the proposed method achieved an average performance score of 60.1, significantly outperforming the strong "Adapter+LAMOL" baseline (56.2) and standard fine-tuning (28.1).
- Parameter Efficiency: The model is remarkably lean. For similar tasks, it achieved better results than standard adapters while using only 2.44M learnable parameters, compared to the baseline's 8.95M.
- Layer-wise Granularity: The study found that task similarity is not binary. A new task might share low-level syntactic patterns with one task but high-level semantic patterns with another. The framework allows the model to compose a unique path for every task. Interestingly, the authors observed that higher layers in the Transformer are more task-specific, while lower layers are more frequently reused across tasks.
- Forward Transfer: By reusing modules, the model naturally facilitates "forward transfer," where knowledge from previous tasks makes it faster and easier to learn subsequent ones.
Conclusion and Future Work
This framework demonstrates that adaptive composition is a powerful tool for continual sequence generation. By allowing the model to decide when to grow and when to share, it achieves state-of-the-art performance with high parameter efficiency.
While the "Decision Stage" currently adds some training overhead (approximately 6 epochs), the modules can be computed in parallel to mitigate this. Future research will look into removing the requirement for a "task-id" during inference by using an autonomous task-classifier and exploring how this compositional approach scales to other NLP problems like machine translation and text classification.
Summary updated