Meta-Semantics Augmented Few-Shot Relational Learning

Han Wu, Jie YinEMNLP 2025

TL;DR

This paper introduces PromptMeta, a novel meta-learning framework for few-shot relational learning in knowledge graphs (KGs). PromptMeta effectively integrates "meta-semantics" – high-level patterns learned across tasks – with relational information to significantly improve the ability to predict missing links in KGs, even with very limited training data.

What Problem Does This Paper Address?

Knowledge graphs (KGs) are powerful tools for representing real-world information, but they are often incomplete and suffer from a "long-tail" distribution where many relations have very few examples. This makes it challenging to predict missing relationships, especially for rare or newly emerging ones. Few-shot relational learning (FSRL) aims to tackle this by learning from a small number of examples. While existing FSRL methods primarily focus on the structural relationships within KGs, they often overlook the rich semantic information inherent in the entities and relations. This paper addresses the gap of effectively leveraging this semantic information to enhance generalization in few-shot relational learning.

Methods

PromptMeta introduces a novel meta-learning framework with three core components:

  • Context-Aware Neighbor Aggregation: This component enriches entity embeddings by considering their local neighborhood information. It uses an attention mechanism to weigh the importance of neighboring relation-entity tuples, allowing for a more nuanced representation of entities.
  • Learnable Meta-Semantic Prompt Pool (MSP): This is a key innovation. Instead of just using relational information, PromptMeta learns and stores "meta-semantic prompts" – high-level semantic patterns that are shared across different tasks (relations). These prompts are dynamically updated during meta-training to capture transferable knowledge. For a given task, a task-semantic embedding is computed from the support set, and the most relevant meta-semantic prompt is retrieved from the MSP pool.
  • Relational and Meta-Semantic Information Fusion: The framework fuses the task-relational embedding (derived from entity and relation embeddings) with the retrieved meta-semantic prompt using a learnable fusion token. This combined representation, called the meta-representation, is then used for prediction.

The entire framework is optimized jointly within a meta-learning paradigm, allowing for adaptation to new relations with limited supervision.

Datasets

The authors evaluate PromptMeta on two widely used KG benchmarks for FSRL:

  • Nell-One: A benchmark KG dataset.
  • Wiki-One: Another benchmark KG dataset.

The splits used for training, validation, and testing relations follow standard experimental setups from prior work.

Key Insights and Ideas

  • Semantics Matter: The paper strongly argues that semantic information, beyond just structural relationships, is crucial for effective few-shot relational learning.
  • Meta-Semantics for Transfer: The concept of a "Meta-Semantic Prompt Pool" is novel. It allows the model to learn abstract semantic patterns that can be transferred across different relations, improving generalization.
  • Dynamic Fusion: The learnable fusion mechanism allows for flexible integration of task-specific relational information with generalizable meta-semantic knowledge, tailored to each task.
  • BERT Embeddings Boost Performance: Using BERT-pretrained semantic embeddings significantly outperforms GloVe embeddings, highlighting the benefit of richer semantic representations.

Key Results

PromptMeta consistently outperforms state-of-the-art FSRL methods on both Nell-One and Wiki-One datasets across 1-shot and 5-shot settings. For instance, on Nell-One in the 1-shot setting, PromptMeta surpasses the second-best method (HiRe) by 1.7% in MRR and 5.4% in Hits@1. The performance gains are even more pronounced in the 5-shot setting, where PromptMeta achieves 10.5% higher MRR and 15.5% higher Hits@1. Ablation studies confirm that each component of PromptMeta (context-aware neighbor aggregation, MSP pool, and fusion mechanism) contributes significantly to the overall performance.

Limitations and Future Work

  • Dependence on Related Relations: The effectiveness of the MSP pool depends on the presence of semantically related relations during meta-training. If trained on highly diverse or unrelated relations, the model's ability to learn generalizable meta-semantics might be limited.
  • Quality of Pretrained Embeddings: The performance is influenced by the quality of pretrained semantic embeddings. Future work could explore more advanced pretrained language models.
  • Memory Cost: Incorporating semantic information adds to the memory cost. Future research could focus on memory-efficient solutions for larger-scale KGs.

Summary updated

BibTeX
@inproceedings{wu2025meta,
  author = {Han Wu and Jie Yin},
  title = {{Meta-Semantics Augmented Few-Shot Relational Learning}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
}
Figure