Abstract
Most research on lifelong learning applies to images or games, but not language. We present LAMOL, a simple yet effective method for lifelong language learning (LLL) based on language modeling. LAMOL replays pseudo-samples of previous tasks while requiring no extra memory or model capacity. Specifically, LAMOL is a language model that simultaneously learns to solve the tasks and generate training samples. When the model is trained for a new task, it generates pseudo-samples of previous tasks for training alongside data for the new task. The results show that LAMOL prevents catastrophic forgetting without any sign of intransigence and can perform five very different language tasks sequentially with only one model. Overall, LAMOL outperforms previous methods by a considerable margin and is only 2-3% worse than multitasking, which is usually considered the LLL upper bound.
The Challenge of Catastrophic Forgetting
In standard machine learning, models suffer from catastrophic forgetting: when trained on a new task, they abruptly lose the ability to perform previously learned ones. This happens because the data distribution shifts, biasing the model's weights toward the most recent information.
Traditional approaches to Lifelong Learning (LLL) typically involve regularization (constraining weight updates), architectural changes (adding modules), or data-based replay (storing old data). However, these often struggle with fundamentally different NLP tasks or incur significant memory overhead. LAMOL addresses this by leveraging the generative power of language models to perform "pseudo-replay."
The LAMOL Framework: Language Modeling for Replay
The core innovation of LAMOL is that a single language model acts as both the solver and the data generator.
1. Unified Task Format
LAMOL treats every NLP task as a Question Answering (QA) problem. Whether it's sentiment analysis or semantic parsing, the model receives a context and a question and is trained to generate an answer. This allows a single GPT-2 model to handle diverse tasks using a unified generative objective.
2. Pseudo-Replay Mechanism
Instead of saving real samples from previous tasks, LAMOL trains the model to be a generator.
- Training: When learning Task 1, the model learns to solve the task and to generate the training samples themselves when prompted with a special
GENtoken. - Replay: When moving to Task 2, the model uses its internal "memory" to generate "pseudo-samples" of Task 1. It then trains on a mixture of these "imagined" Task 1 samples and the real Task 2 data.
3. Task-Specific Tokens
To prevent the model from "forgetting how to imagine" the earliest tasks in a long sequence, the authors introduced task-specific tokens (e.g., [SQuAD], [SST]). This ensures the model can specifically target and generate a balanced distribution of pseudo-samples for every previous task, stabilizing internal memory across long sequences.
Evaluation and Results
The authors evaluated LAMOL on nine diverse datasets, including the decaNLP subset (SQuAD, WikiSQL, SST, QA-SRL, WOZ) and various text classification tasks (AGNews, Amazon, DBPedia, Yahoo, Yelp). These represent a significant challenge as they require vastly different types of reasoning and output formats.
Key findings include:
- Near-Optimal Performance: LAMOL outperformed state-of-the-art LLL methods (like EWC, MAS, and GEM) by a wide margin, coming within 2–3% of the "Multitask Learning" upper bound.
- Zero Intransigence: Unlike regularization methods, LAMOL remains capable of learning new tasks effectively without becoming "stiff."
- Efficiency: The method is robust even with limited model capacity, achieving high performance using the smallest GPT-2 model.
Discussion and Future Work
LAMOL demonstrates that generative models can serve as their own implicit memory. While pseudo-replay is highly scalable and requires zero extra storage, it does introduce additional computational costs during training. The authors also noted that as tasks increase in complexity, the quality of pseudo-samples can degrade—a phenomenon termed "chaos." Future research will explore the limits of this scaling and how to maintain sample quality over hundreds of sequential tasks.
Summary updated