Context-Aware Adapter Tuning for Few-Shot Relation Learning in Knowledge Graphs

Ran Liu, Zhongzhou Liu, Xiao-Li Li, Yuan Fang

TL;DR

This paper introduces RelAdapter, a novel framework for few-shot relation learning in knowledge graphs. It tackles the challenge of adapting meta-learned knowledge to new, unseen relations by employing a context-aware adapter that leverages both relation-specific tuning and contextual information from the knowledge graph, leading to improved performance.

What Problem Does This Paper Address?

Knowledge graphs (KGs) are powerful tools for representing real-world facts, but they often suffer from incompleteness, meaning many relations are missing. Predicting these missing relations, especially for new or rare ones with very few examples (few-shot learning), is a significant challenge. Existing methods often rely on meta-learning, which trains a model to quickly adapt to new tasks. However, a key assumption in meta-learning is that the tasks seen during training (base relations) and the tasks encountered during testing (novel relations) come from the same underlying distribution (i.i.d. assumption). The authors argue that this assumption often doesn't hold in practice, as different relations can have vastly different characteristics, leading to performance degradation when adapting to novel relations. They demonstrate this by showing a wide variance in the similarity between relations in benchmark KGs.

Methods

RelAdapter addresses this problem with two main innovations:

  • Context-Aware Adapter Module: This is a lightweight, parameter-efficient module that sits between the meta-learned knowledge (prior) and the specific task at hand. It allows for fine-grained, tunable adaptation of the meta-knowledge to each individual novel relation. This is inspired by parameter-efficient fine-tuning techniques. The adapter uses a bottleneck structure to reduce the number of parameters, making it less prone to overfitting on limited few-shot data.
  • Contextual Information Enrichment: To further enhance adaptation, RelAdapter injects contextual information about the target relation. This context is extracted from the KG's structure (e.g., neighboring entities) without requiring additional manual annotation. This augmented information helps the adapter understand the specific nuances of the novel relation, leading to more tailored adaptation.

The overall framework integrates these components into a meta-learning pipeline, consisting of meta-training (learning a general prior) and meta-testing (adapting to novel relations using the adapter and contextual information).

Datasets

The authors evaluate RelAdapter on three standard benchmark KG datasets:

  • WIKI: A large-scale KG.
  • FB15K-237: A subset of Freebase, specifically curated to remove inverse relations that can cause data leakage.
  • UMLS: A biomedical KG.

The datasets are split into pre-training, training, validation, and test sets. For the smaller UMLS dataset, relations with fewer than 50 triplets were removed.

Key Insights and Ideas

  • The i.i.d. assumption is a bottleneck: The paper empirically demonstrates that the assumption of independent and identically distributed tasks in meta-learning is a significant limitation for few-shot relation learning in KGs, as relations can be quite diverse.
  • Relation-specific adaptation is crucial: Simply applying a generic adaptation strategy to all novel relations is insufficient. The adapter module's ability to tune itself for each relation is key to overcoming distribution shifts.
  • Context matters: Leveraging the surrounding KG structure (entity contexts) provides valuable information that significantly aids in adapting to novel relations, especially when combined with the adapter.
  • Parameter efficiency is important for few-shot learning: The lightweight nature of the adapter module prevents overfitting on the limited data available in few-shot scenarios.

Key Results

RelAdapter consistently outperforms state-of-the-art baselines across all three benchmark datasets in the 3-shot setting. Specifically:

  • It achieves an average improvement of 20.1% in MRR and 15.1% in Hit@10 compared to MetaR, its closest competitor.
  • The adapter module is highly parameter-efficient, requiring a negligible number of parameters compared to models like MetaR.
  • The runtime efficiency is comparable to existing methods, demonstrating that significant performance gains can be achieved without a prohibitive increase in computational cost.

Limitations and Future Work

  • Framework integration: The current RelAdapter framework is integrated with the MetaR meta-learning approach. The authors suggest that exploring its integration with other meta-learning frameworks would be a valuable future direction.
  • Dataset sensitivity: While RelAdapter performs well, the paper notes that for smaller datasets like UMLS, the performance can be more sensitive to hyperparameters and the introduction of noise from broader contextual information. Future work could explore more robust context aggregation strategies for smaller or more specialized KGs.
  • Number of hops for context: The study shows that while context is beneficial, using too many hops (e.g., 3-hop vs. 1-hop) can sometimes degrade performance, especially on smaller datasets, due to potential noise. Further research into optimal context selection strategies could be beneficial.

Summary updated

Abstract

Knowledge graphs (KGs) are instrumental in various real-world applications, yet they often suffer from incompleteness due to missing relations. To predict instances for novel relations with limited training examples, few-shot relation learning approaches have emerged, utilizing techniques such as meta-learning. However, the assumption is that novel relations in meta-testing and base relations in meta-training are independently and identically distributed, which may not hold in practice. To address the limitation, we propose RelAdapter, a context-aware adapter for few-shot relation learning in KGs designed to enhance the adaptation process in meta-learning. First, RelAdapter is equipped with a lightweight adapter module that facilitates relation-specific, tunable adaptation of meta-knowledge in a parameter-efficient manner. Second, RelAdapter is enriched with contextual information about the target relation, enabling enhanced adaptation to each distinct relation. Extensive experiments on three benchmark KGs validate the superiority of RelAdapter over state-of-the-art methods.
BibTeX
@inproceedings{liu2024context,
  author = {Ran Liu and Zhongzhou Liu and Xiao-Li Li and Yuan Fang},
  title = {{Context-Aware Adapter Tuning for Few-Shot Relation Learning in Knowledge Graphs}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=TZztJx4neK},
}
Figure