Introduction: The Challenge of Continual Few-shot Learning
In dynamic real-world environments, Relation Extraction (RE) systems must adapt to new relation types (e.g., born_in, subsidiary_of) as they emerge. Traditional Continual Relation Learning (CRL) assumes that ample labeled data is available for every new task. However, in practice, new relations often appear with only a handful of examples—a scenario known as Few-shot Learning.
The intersection of these fields, Continual Few-shot Relation Learning (CFRL), presents a "double trouble" scenario: learning new relations from minimal data while preventing the model from overwriting weights optimized for previous tasks (catastrophic forgetting).
Methodology: The ERDA Framework
To address these challenges, the authors introduce ERDA (Embedding space Regularization and Data Augmentation). ERDA utilizes a Siamese encoder architecture and is built upon three core pillars:
1. Embedding Space Regularization
Rather than relying solely on standard cross-entropy, ERDA employs two margin-based losses to structure the embedding space. A Multi-margin loss forces samples of the same relation to cluster tightly while pushing different relations apart. A Pairwise margin loss specifically penalizes the model when the "closest incorrect" relation is too semantically similar to the ground truth.
2. Hard Negative Contrastive Learning
To mitigate forgetting with minimal memory overhead (storing only one sample per relation), ERDA generates "hard negatives" on the fly. By swapping an entity in a memory sample with a random entity from the current batch, the model is forced to focus on the relational context rather than memorizing specific entity pairs.
3. Self-Supervised Data Augmentation
To combat data scarcity in few-shot tasks, ERDA "hunts" for additional examples in an unlabeled corpus like Wikipedia. It employs Entity Matching and Similarity Search (using Faiss) to expand the training set before the main learning phase begins.
Key Results and Visual Proof
ERDA consistently outperformed state-of-the-art methods like EMAR and IDLVQ-C across multiple benchmarks. For instance, on the FewRel 10-way 5-shot benchmark, ERDA achieved 53.23% accuracy after 8 tasks, a nearly 13% improvement over the previous best method.
The effectiveness of the embedding space regularization is most evident in t-SNE visualizations. As shown below, ERDA maintains significantly tighter clusters and larger inter-class margins compared to IDLVQ-C, even after multiple rounds of sequential learning.
Conclusion
ERDA demonstrates that structuring the embedding space into distinct, compact "territories" is critical for successful continual learning under data scarcity. By combining regularization with targeted data augmentation, the framework effectively balances the need to learn new information with the preservation of existing knowledge.
Summary updated