Abstract
Continual learning in text generation requires models to learn a stream of tasks without suffering from catastrophic forgetting. This paper introduces a framework that adaptively decides whether to reuse existing neural modules or add new ones for each incoming task. By integrating modular flexibility with pseudo-experience replay, the proposed method achieves state-of-the-art performance and superior parameter efficiency compared to standard adapter-based approaches.
Introduction
In real-world applications, AI models must continuously learn new tasks over time. However, standard neural networks often experience "catastrophic forgetting," where learning a new task (Task B) overwrites the knowledge required for a previous one (Task A). Current strategies typically fall into two categories: parameter sharing and parameter isolation. Parameter sharing encourages knowledge transfer but is highly susceptible to forgetting. Conversely, parameter isolation (e.g., using dedicated adapters for each task) prevents forgetting but inhibits knowledge transfer and leads to redundant, oversized models as tasks accumulate.
The authors propose a "middle way": a model that reuses existing modules when tasks are similar and adds new ones only when necessary, maintaining efficiency and performance across a sequence of tasks.
Methodology
The framework employs a two-stage approach integrated into a Transformer architecture, focusing on layer-wise modularity.
The Decision Stage
Before training on a new task, the model undergoes a "test run" using Hidden State Mixing. It evaluates existing adapter modules alongside a newly initialized one by calculating a weighted average of their outputs. An entropy loss is applied to penalize "indecisiveness," forcing the model to determine whether an existing module is sufficient or if a new one must be added. Smart initialization is used to favor existing modules, ensuring new capacity is only created when necessary.
The Training Stage
Once the architecture is composed for the new task, training begins. To mitigate forgetting in shared modules, the authors utilize Pseudo Experience Replay. Instead of storing sensitive or bulky historical data, the model generates synthetic examples of previous tasks to reinforce its prior knowledge during the current training phase.
Results and Key Insights
The framework was evaluated on eight task sequences, ranging from similar domains (e.g., restaurant and hotel NLG) to highly dissimilar tasks (e.g., SQL generation and news summarization).
- Performance and Efficiency: On similar tasks, the method achieved an average score of 66.0, outperforming standard adapters (64.2). In dissimilar scenarios, it surpassed the "Adapter+CL" baseline while using significantly fewer parameters (6.60M vs. 8.95M).
- Layer-wise Specialization: A key finding was that not all layers behave the same. Lower layers often capture general syntax and are frequently reused, while higher layers tend to be more task-specific. This granular composition allows for a "mix-and-match" approach that is more powerful than monolithic task adapters.
- Robustness: The model maintained high performance as the task sequence grew, demonstrating resilience against the interference and capacity issues that plague other models.
Conclusion
This adaptive compositional approach offers a scalable solution for continual sequence generation. While it introduces some training overhead during the decision stage and currently requires task identity during inference, it provides a robust foundation for building AI systems that learn incrementally. Future work will explore automatic task detection and the generalization of this modular approach to other domains like text classification and machine translation.
Summary updated