Abstract
The ability to continuously expand knowledge over time and utilize it to rapidly generalize to new tasks is a key feature of human linguistic intelligence. Existing models that pursue rapid generalization to new tasks (e.g., few-shot learning methods), however, are mostly trained in a single shot on fixed datasets, unable to dynamically expand their knowledge; while continual learning algorithms are not specifically designed for rapid generalization. We present a new learning setup, Continual Learning of Few-Shot Learners (CLIF), to address the challenges of both learning settings in a unified setup. CLIF assumes a model learns from a sequence of diverse NLP tasks arriving sequentially, accumulating knowledge for improved generalization to new tasks, while also retaining performance on the tasks learned earlier. We examine how the generalization ability is affected in the continual learning setup, evaluate a number of continual learning algorithms, and propose a novel regularized adapter generation approach. We find that catastrophic forgetting affects generalization ability to a less degree than performance on seen tasks; while continual learning algorithms can still bring considerable benefit to the generalization ability.
Introduction: The Need for Lifelong Knowledge Accumulation
In the current NLP landscape, two powerful but isolated paradigms dominate: Few-Shot Learning (FSL) and Continual Learning (CL). FSL models, like GPT-3, can learn new tasks from just a few examples but are typically trained on static, "frozen" datasets, lacking the ability to grow their knowledge base over time. Conversely, CL focuses on incremental learning, but its primary goal is usually retaining performance on old tasks (avoiding catastrophic forgetting) rather than becoming a better learner for future, unseen tasks.
Human intelligence, however, does both: we learn incrementally, and each new concept we master makes us faster at picking up the next related one. This paper introduces CLIF (Continual LearnIng of Few-shot Learners), a framework that challenges AI models to learn from a sequence of tasks while simultaneously improving their ability to generalize to new tasks with minimal data.
Methodology: BiHNet+Reg
The researchers built their approach on top of the BART language model using parameter-efficient Adapters—small, trainable modules inserted into transformer layers while the rest of the model remains frozen. Their novel method, BiHNet+Reg (Bi-level Hypernetworks for Adapters with Regularization), introduces three key components:
- Context Predictor: Generates two types of task representations: "high-resource" (using all available data) and "few-shot" (mimicking the target scenario).
- Hypernetwork: Instead of training adapters directly, a separate hypernetwork learns to generate the weights for these adapters based on the task representation.
- Regularization: To prevent the hypernetwork from forgetting how to handle old tasks, the authors apply an L2 distance penalty that discourages the network from changing the weights it would generate for previously seen tasks.
Experimental Setup and Benchmarks
To evaluate this setup, the authors introduced two comprehensive benchmarks:
- CLIF-26: Uses 9 tasks from the GLUE benchmark for the "upstream" continual learning phase and 17 diverse tasks from the DivFSL suite for the "few-shot" evaluation.
- CLIF-55: A massive scale-up involving 45 upstream tasks and 10 few-shot tasks, covering sentiment analysis, paraphrasing, hate speech detection, and fact-checking.
Key Insights and Results
Forgetting is Asymmetric
A surprising finding is that "catastrophic forgetting" hurts performance on seen tasks much more than it hurts the model's ability to generalize to new ones. Even if a model forgets the specific mechanics of Task A, the underlying "meta-knowledge" gained often still assists in learning Task B faster.
Performance Gains
The proposed BiHNet+Reg outperformed existing baselines significantly:
- Retention: On CLIF-26, it achieved a Final Accuracy of 77.22%, compared to just 19.73% for a vanilla sequential BART model.
- Generalization: It showed a 1.8% relative improvement in few-shot accuracy over zero-knowledge baselines. In contrast, vanilla sequential training often resulted in "negative transfer," where learning more tasks actually made the model worse at few-shot learning (-3.3%).
- Efficiency: The method is privacy-friendly; unlike "replay-based" CL methods, it doesn't need to store actual examples from old tasks, only their compact vector representations.
Limitations and Future Work
While the adapter-based approach is highly efficient, it currently has a lower performance ceiling than full-model fine-tuning due to fewer parameters. Future research will explore task-agnostic learning (moving away from explicit task boundaries), utilizing unlabeled data to refine accumulated knowledge, and scaling hypernetwork capacity to match full-parameter fine-tuning performance.
Summary updated