Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning

Xisen Jin, Bill Yuchen Lin, Mohammad Rostami, Xiang Ren

Abstract

The ability to continuously expand knowledge over time and utilize it to rapidly generalize to new tasks is a key feature of human linguistic intelligence. Existing models that pursue rapid generalization to new tasks (e.g., few-shot learning methods), however, are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge; while continual learning algorithms are not specifically designed for rapid generalization. We present a new learning setup, Continual Learning of Few-Shot Learners (CLIF), to address the challenges of both learning settings in a unified setup. CLIF assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks, while also retaining performance on the tasks learned earlier. We examine how the generalization ability is affected in the continual learning setup, evaluate a number of continual learning algorithms, and propose a novel regularized adapter generation approach. We find that catastrophic forgetting affects generalization ability to a less degree than performance on seen tasks; while continual learning algorithms can still bring considerable benefit to the generalization ability.

Introduction: The Need for Lifelong Knowledge Accumulation

In the current NLP landscape, two powerful but isolated paradigms dominate: Few-Shot Learning (FSL) and Continual Learning (CL). FSL models, like GPT-3, can learn new tasks from just a few examples but are typically trained on static, "frozen" datasets, lacking the ability to grow their knowledge base over time. Conversely, CL focuses on incremental learning, but its primary goal is usually retaining performance on old tasks (avoiding catastrophic forgetting) rather than becoming a better learner for future, unseen tasks.

Human intelligence, however, does both: we learn incrementally, and each new concept we master makes us faster at picking up the next related one. This paper introduces CLIF (Continual LearnIng of Few-shot Learners), a framework that challenges AI models to learn from a sequence of tasks while simultaneously improving their ability to generalize to new tasks with minimal data.

The CLIF framework involves training on a sequence of tasks (e.g., GLUE) and evaluating the model's ability to predict on seen tasks and adapt to new, unseen tasks with few-shot adaptation.

Methodology: BiHNet+Reg

The researchers built their approach on top of the BART language model using parameter-efficient Adapters—small, trainable modules inserted into transformer layers while the rest of the model remains frozen. Their novel method, BiHNet+Reg (Bi-level Hypernetworks for Adapters with Regularization), introduces three key components:

  1. Context Predictor: Generates two types of task representations: "high-resource" (using all available data) and "few-shot" (mimicking the target scenario).
  2. Hypernetwork: Instead of training adapters directly, a separate hypernetwork learns to generate the weights for these adapters based on the task representation.
  3. Regularization: To prevent the hypernetwork from forgetting how to handle old tasks, the authors apply an L2 distance penalty that discourages the network from changing the weights it would generate for previously seen tasks.
Comparison of Vanilla CL, Memory-Based CL (MbPA++), and the proposed HyperCL (BiHNet+Reg) which utilizes bi-level task representations and adapter generation.

Experimental Setup and Benchmarks

To evaluate this setup, the authors introduced two comprehensive benchmarks:

  • CLIF-26: Uses 9 tasks from the GLUE benchmark for the "upstream" continual learning phase and 17 diverse tasks from the DivFSL suite for the "few-shot" evaluation.
  • CLIF-55: A massive scale-up involving 45 upstream tasks and 10 few-shot tasks, covering sentiment analysis, paraphrasing, hate speech detection, and fact-checking.

Key Insights and Results

Forgetting is Asymmetric

A surprising finding is that "catastrophic forgetting" hurts performance on seen tasks much more than it hurts the model's ability to generalize to new ones. Even if a model forgets the specific mechanics of Task A, the underlying "meta-knowledge" gained often still assists in learning Task B faster.

Performance Gains

The proposed BiHNet+Reg outperformed existing baselines significantly:

  • Retention: On CLIF-26, it achieved a Final Accuracy of 77.22%, compared to just 19.73% for a vanilla sequential BART model.
  • Generalization: It showed a 1.8% relative improvement in few-shot accuracy over zero-knowledge baselines. In contrast, vanilla sequential training often resulted in "negative transfer," where learning more tasks actually made the model worse at few-shot learning (-3.3%).
  • Efficiency: The method is privacy-friendly; unlike "replay-based" CL methods, it doesn't need to store actual examples from old tasks, only their compact vector representations.

Limitations and Future Work

While the adapter-based approach is highly efficient, it currently has a lower performance ceiling than full-model fine-tuning due to fewer parameters. Future research will explore task-agnostic learning (moving away from explicit task boundaries), utilizing unlabeled data to refine accumulated knowledge, and scaling hypernetwork capacity to match full-parameter fine-tuning performance.

Summary updated

Abstract

The ability to continuously expand knowledge over time and utilize it to rapidly generalize to new tasks is a key feature of human linguistic intelligence. Existing models that pursue rapid generalization to new tasks (e.g., few-shot learning methods), however, are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge; while continual learning algorithms are not specifically designed for rapid generalization. We present a new learning setup, Continual Learning of Few-Shot Learners (CLIF), to address the challenges of both learning settings in a unified setup. CLIF assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks, while also retaining performance on the tasks learned earlier. We examine how the generalization ability is affected in the continual learning setup, evaluate a number of continual learning algorithms, and propose a novel regularized adapter generation approach. We find that catastrophic forgetting affects generalization ability to a less degree than performance on seen tasks; while continual learning algorithms can still bring considerable benefit to the generalization ability.
BibTeX
@inproceedings{jin2021learn,
  author = {Xisen Jin and Bill Yuchen Lin and Mohammad Rostami and Xiang Ren},
  title = {{Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.62},
}
Figure