Context-Aware Adapter Tuning for Few-Shot Relation Learning in Knowledge Graphs

Liu Ran, Zhongzhou Liu, Xiaoli Li, Yuan Fang

Abstract

Knowledge graphs (KGs) are instrumental in various real-world applications, yet they often suffer from incompleteness due to missing relations. To predict instances for novel relations with limited training examples, few-shot relation learning approaches have emerged, utilizing techniques such as meta-learning. However, the assumption is that novel relations in meta-testing and base relations in meta-training are independently and identically distributed, which may not hold in practice. To address the limitation, we propose RelAdapter, a context-aware adapter for few-shot relation learning in KGs designed to enhance the adaptation process in meta-learning. First, RelAdapter is equipped with a lightweight adapter module that facilitates relation-specific, tunable adaptation of meta-knowledge in a parameter-efficient manner. Second, RelAdapter is enriched with contextual information about the target relation, enabling enhanced adaptation to each distinct relation. Extensive experiments on three benchmark KGs validate the superiority of RelAdapter over state-of-the-art methods.

Introduction

Knowledge Graphs (KGs) like Wikidata or Freebase are dynamic entities where new relations emerge constantly, often accompanied by only a handful of examples—a phenomenon known as the "long tail" problem. To address this, Few-Shot Relation Learning (FSRL) has become a vital area of research, primarily leveraging meta-learning to predict missing links for these sparse relations.

Standard meta-learning frameworks attempt to learn a global prior—a set of initial parameters that can be rapidly fine-tuned for any new task. However, these methods typically rely on the assumption that training and testing tasks are independently and identically distributed (i.i.d.). In the context of KGs, this assumption is often violated. Relations vary significantly in their semantics and distributions; for instance, a model optimized for "place of birth" may not generalize well to "chemical composition." RelAdapter is designed to overcome this limitation by enabling rapid, relation-specific specialization.

The RelAdapter Framework

RelAdapter enhances the standard meta-learning pipeline (specifically the MetaR framework) through two primary innovations: a model-level adapter and data-level context awareness.

RelAdapter architecture overview showing the adapter module, entity context aggregation, and the meta-testing pipeline.
  1. The Adapter Module (Model-level): Instead of relying solely on global gradient updates, RelAdapter introduces a lightweight "adapter"—a bottleneck feed-forward network with a residual connection. During meta-testing, this adapter is initialized and tuned specifically for the target relation. Its minimal parameter count allows for effective optimization on as few as 1–5 examples without the risk of overfitting.
  2. Context-Aware Adaptation (Data-level): To bolster the signal from limited data, RelAdapter augments entity embeddings with information from their local neighborhood. For example, when learning about a "Bird," the model incorporates features from neighbors like "Feathers" and "Beaks." This contextual augmentation helps the adapter better understand the semantic neighborhood of a novel relation.

Experimental Setup

The model was evaluated on three benchmark KGs: WIKI (derived from Wikidata), FB15K-237 (Freebase), and UMLS (medical concepts). The experiments followed standard splits, ensuring that relations encountered during testing were entirely absent from the training phase.

Analysis of Relation Distributions

A core motivation for RelAdapter is the empirical observation that the i.i.d. assumption is fundamentally flawed in KGs. The authors analyzed the cosine similarity between different relation pairs across the benchmark datasets.

Cosine similarity distributions of relation pairs across WIKI, FB15K-237, and UMLS datasets, highlighting the non-i.i.d. nature of KG relations.

The results demonstrate significant variance in relation similarities, proving that a "one-size-fits-all" global prior is insufficient. RelAdapter's parameter-efficient design (accounting for only 0.002% of total parameters on the WIKI dataset) allows it to bridge these distributional gaps effectively.

Key Results and Insights

RelAdapter consistently outperformed state-of-the-art (SOTA) baselines, including HiRe, GANA, and MetaR, across all evaluated metrics (MRR, Hit@10, Hit@1).

  • Significant Performance Gains: On the WIKI dataset, RelAdapter improved the Mean Reciprocal Rank (MRR) by nearly 10% over the strongest baseline. On the specialized UMLS dataset, it achieved an MRR of 0.608, a substantial increase from MetaR’s 0.435.
  • Scaling with Data: RelAdapter demonstrated superior scalability as the number of available examples ($K$) increased from 1 to 10, indicating more efficient utilization of additional supervision.
  • Fresh Adaptation: Surprisingly, the authors found that randomly initializing the adapter for each new task during testing outperformed transferring a pre-trained adapter. This suggests that "fresh" adaptation is more effective at overcoming the distribution shift between seen and unseen relations.

Conclusion

RelAdapter addresses the critical challenge of distribution shift in few-shot relation learning for KGs. By combining lightweight adapters with contextual entity information, it provides a robust and efficient mechanism for adapting to novel relations. Future work includes generalizing the adapter logic to other meta-learning architectures and exploring automated hyperparameter tuning.

Summary updated

Abstract

Knowledge graphs (KGs) are instrumental in various real-world applications, yet they often suffer from incompleteness due to missing relations. To predict instances for novel relations with limited training examples, few-shot relation learning approaches have emerged, utilizing techniques such as meta-learning. However, the assumption is that novel relations in meta-testing and base relations in meta-training are independently and identically distributed, which may not hold in practice. To address the limitation, we propose RelAdapter, a context-aware adapter for few-shot relation learning in KGs designed to enhance the adaptation process in meta-learning. First, RelAdapter is equipped with a lightweight adapter module that facilitates relation-specific, tunable adaptation of meta-knowledge in a parameter-efficient manner. Second, RelAdapter is enriched with contextual information about the target relation, enabling enhanced adaptation to each distinct relation. Extensive experiments on three benchmark KGs validate the superiority of RelAdapter over state-of-the-art methods.
BibTeX
@inproceedings{ran2024context,
  author = {Liu Ran and Zhongzhou Liu and Xiaoli Li and Yuan Fang},
  title = {{Context-Aware Adapter Tuning for Few-Shot Relation Learning in Knowledge Graphs}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=TZztJx4neK},
}
Figure