Lifelong Learning of Few-shot Learners across NLP Tasks

Xisen Jin, Mohammad Rostami, Xiang Ren

Abstract

The ability to continuously expand knowledge over time and utilize it to rapidly generalize to new tasks is a key feature of human linguistic intelligence. Existing models that pursue rapid generalization to new tasks (e.g., few-shot learning methods), however, are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge; while continual learning algorithms are not specifically designed for rapid generalization. We present a new learning setup, Continual Learning of Few-Shot Learners (CLIF), to address the challenges of both learning settings in a unified setup. CLIF assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks, while also retaining performance on the tasks learned earlier. We examine how the generalization ability is affected in the continual learning setup, evaluate a number of continual learning algorithms, and propose a novel regularized adapter generation approach. We find that catastrophic forgetting affects generalization ability to a less degree than performance on seen tasks; while continual learning algorithms can still bring considerable benefit to the generalization ability.

Introduction: The CLIF Framework

In modern NLP, two paradigms dominate the landscape of efficient learning: Few-shot Learning (FSL) and Continual Learning (CL). FSL focuses on rapid generalization to new tasks with minimal data but typically operates in a static, offline environment. Conversely, CL addresses the sequential acquisition of tasks, primarily focusing on stability—preventing the "catastrophic forgetting" of old knowledge.

This paper introduces CLIF (Continual Learning of Few-shot Learners), a framework that bridges these paradigms. CLIF requires a model to learn from a stream of diverse tasks, accumulating knowledge that enhances its ability to master future, unseen tasks with very few examples, all while maintaining performance on previously encountered data.

Overview of the CLIF framework, distinguishing between training on a sequence of tasks and evaluating on both seen and new tasks.

Methodology: BiHNet+Reg

The authors utilize BART as a base sequence-to-sequence model, augmented with Adapters—small, efficient modules that allow for task-specific tuning without updating the entire model. The core innovation is BiHNet+Reg (Bi-level Hypernetworks with Regularization).

Unlike standard methods that train adapter weights directly, BiHNet uses a Hypernetwork to generate these weights based on task representations. A Context Predictor creates two versions of these representations: a "high-resource" version (using all data) and a "few-shot" version (using only $K$ examples). This bi-level training forces the model to remain robust in low-data regimes. To ensure stability, a regularization component stores previous task representations and penalizes the hypernetwork if it deviates from previously learned weight-generation patterns.

Architectural comparison between Vanilla CL, Memory-Based CL, and the proposed HyperCL (BiHNet+Reg) approach.

Experimental Setup and Results

The framework was evaluated on two benchmarks: CLIF-26 (9 upstream GLUE tasks, 17 downstream DivFSL tasks) and CLIF-55 (45 upstream tasks, 10 downstream tasks). All tasks were unified into a sequence-to-sequence question-answering format.

Key Findings:

  1. Generalization vs. Forgetting: A critical insight is that catastrophic forgetting impacts generalization to a lesser degree than it impacts performance on previously seen tasks. Even if a model "forgets" specific task labels, it often retains the underlying linguistic features beneficial for future learning.
  2. The Role of Task Order: The sequence of tasks matters. Encountering highly relevant tasks early in the stream provides a stronger foundation for subsequent few-shot adaptation.
  3. Performance Gains: BiHNet+Reg demonstrated superior stability and generalization. In CLIF-26, it maintained a 77.22% accuracy on seen tasks, whereas a vanilla BART model collapsed to 19.73%. Furthermore, it consistently outperformed standard CL baselines like EWC and MbPA++ in few-shot generalization, particularly in extremely low-data settings ($k=4$).

Conclusion

CLIF presents a more realistic and challenging setting for developing versatile NLP models. By combining bi-level hypernetworks with targeted regularization, the BiHNet+Reg approach successfully balances the need for stability (retaining old knowledge) with plasticity (improving future learning). Future work will explore task-agnostic scenarios and the integration of unlabeled data to further refine lifelong learning capabilities.

Summary updated

Abstract

The ability to continuously expand knowledge over time and utilize it to rapidly generalize to new tasks is a key feature of human linguistic intelligence. Existing models that pursue rapid generalization to new tasks (e.g., few-shot learning methods), however, are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge; while continual learning algorithms are not specifically designed for rapid generalization. We present a new learning setup, Continual Learning of Few-Shot Learners (CLIF), to address the challenges of both learning settings in a unified setup. CLIF assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks, while also retaining performance on the tasks learned earlier. We examine how the generalization ability is affected in the continual learning setup, evaluate a number of continual learning algorithms, and propose a novel regularized adapter generation approach. We find that catastrophic forgetting affects generalization ability to a less degree than performance on seen tasks; while continual learning algorithms can still bring considerable benefit to the generalization ability.
BibTeX
@inproceedings{jin2021lifelong,
  author = {Xisen Jin and Mohammad Rostami and Xiang Ren},
  title = {{Lifelong Learning of Few-shot Learners across NLP Tasks}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.62},
}
Figure