Prototype-Enhanced Few-Shot Relation Extraction Method Based on Cluster Loss Optimization

Shenyi Qian, Bowen Fu, Chao Liu, Songhe Jin, Tong Sun, Zhen Chen, Daiyi Li, Yifan Sun, Yibing Chen, Yuheng LiSymmetry 2025 · DOI

TL;DR

This paper introduces a novel "Prototype-Enhanced Few-Shot Relation Extraction Method Based on Cluster Loss Optimization" to improve the accuracy of identifying relationships between entities in text, especially when labeled data is scarce. By integrating semantic information more effectively and optimizing prototype representations through a new cluster loss function, the method achieves state-of-the-art performance on benchmark datasets.

What Problem Does This Paper Address?

Relation Extraction (RE) is crucial for tasks like information retrieval and knowledge graph construction. However, traditional RE methods require vast amounts of labeled data, which is expensive and time-consuming to acquire. Few-shot Relation Extraction (FSRE) aims to address this by enabling models to learn new relation types from very few labeled examples. While prototype-based methods have shown promise in FSRE, they struggle with similar relation types, noisy data, and distinguishing nuanced semantic relationships. This paper tackles these challenges by proposing a more robust and semantically aware prototype-based approach.

Methods

The proposed method enhances prototype networks with several key innovations:

  • Prompt Encoder: A specially designed prompt encoder unifies instance and relation information, feeding it into a Large Language Model (LLM) to generate high-quality semantic representations of instances and their prototypes. This leverages the LLM's powerful understanding capabilities.
  • Graph Attention Learning: A graph attention mechanism models the associations between conceptual prototypes and instances. This helps extract relation-specific features while maintaining structural symmetry.
  • Prototype-Level Contrastive Learning: This strategy uses bidirectional feature symmetry to fuse interpretable conceptual prototype features with intra-class common features from instance prototypes, creating enhanced relation representations.
  • Cluster Loss Function: A novel cluster loss function is introduced to guide the model in learning a discriminative metric space. This loss function explicitly enforces intra-class compactness (similar samples are close) and inter-class separability (different classes are far apart), improving the model's ability to distinguish between similar relation types and handle outliers. This is presented as an improvement over traditional triplet loss.

Datasets

The proposed model was evaluated on two benchmark datasets:

  • FewRel 1.0: A widely recognized dataset for few-shot RE, comprising 100 relations, with 64 for training, 16 for validation, and 20 for testing.
  • FewRel 2.0: An extension of FewRel 1.0, designed to evaluate domain adaptation capabilities. It includes data from medical, science, and education technology fields, with a training set from the general domain and validation/test sets from the medical field.

Key Insights and Ideas

  • LLMs as Semantic Powerhouses: The paper highlights the significant advantage of leveraging LLMs for their deep semantic understanding to generate richer representations for instances and prototypes, especially in low-resource settings.
  • Symmetry is Key: The emphasis on "symmetry" throughout the method (e.g., symmetry-aware aggregation, bidirectional feature symmetry) suggests a principled approach to ensure consistent and robust representation learning.
  • Cluster Loss for Better Boundaries: The introduction of a cluster loss function directly addresses the challenge of distinguishing semantically similar relations by explicitly pushing class clusters apart, leading to clearer decision boundaries.
  • Beyond Simple Distance: The limitations of traditional triplet loss and hard sample mining are acknowledged, motivating the development of a cluster loss that is less sensitive to sample selection and focuses on global feature distribution.

Key Results

The proposed model achieved state-of-the-art performance on both FewRel 1.0 and FewRel 2.0 datasets.

  • On FewRel 1.0, the model (Ours (CP)) achieved accuracies of 96.57%/97.25% (5-Way 1-Shot Val/Test) and 94.56%/95.35% (10-Way 1-Shot Val/Test), outperforming several strong baselines.
  • On FewRel 2.0, the model demonstrated remarkable domain adaptability, outperforming standard methods like Proto-BERT and BERT-PAIR by at least 11% accuracy on average.
  • Ablation studies showed that each component (relation information, link prediction loss, prototype-level contrastive loss, and cluster loss) contributes significantly to the overall performance. Excluding relation information alone caused a notable drop in accuracy.

Limitations and Future Work

  • LLM Dependence: The authors acknowledge that the model's strong semantic representation capability relies on LLMs. This dependence on external, large pre-trained models can pose challenges for long-term maintainability due to availability and update cycles.
  • Future Directions: The authors plan to extend the model to diverse and few-shot RE datasets (like CrossFewRel) to verify its universality and robustness. They also aim to explore its deployment in low-resource scenarios such as cross-language tasks and industrial datasets. Addressing the LLM dependence is also a key future research area.

Summary updated

BibTeX
@article{qian2025prototype,
  author = {Shenyi Qian and Bowen Fu and Chao Liu and Songhe Jin and Tong Sun and Zhen Chen and Daiyi Li and Yifan Sun and Yibing Chen and Yuheng Li},
  title = {{Prototype-Enhanced Few-Shot Relation Extraction Method Based on Cluster Loss Optimization}},
  journal = {Symmetry},
  year = {2025},
  doi = {10.3390/sym17101673},
}
Figure