Improving Discriminative Learning for Zero-Shot Relation Extraction

Van-Hien Tran, Hiroki Ouchi, Taro Watanabe, Yūji Matsumoto2022 · DOI

Abstract

Zero-shot relation extraction (ZSRE) aims to predict target relations that cannot be observed during training. While most previous studies have focused on fully supervised relation extraction and achieved considerably high performance, less effort has been made towards ZSRE. This study proposes a new model incorporating discriminative embedding learning for both sentences and semantic relations. In addition, a self-adaptive comparator network is used to judge whether the relationship between a sentence and a relation is consistent. Experimental results on two benchmark datasets showed that the proposed method significantly outperforms the state-of-the-art methods.

Introduction

Relation Extraction (RE) is the task of identifying semantic relationships between entities in a sentence (e.g., in "Paris is the capital of France," the relation is capital_of). While supervised RE works well, it fails when encountering "unseen" relations not present in the training data. Zero-Shot Relation Extraction (ZSRE) attempts to solve this by generalizing from seen relations to unseen ones using semantic descriptions of the relations.

However, previous methods struggled with two main issues: a lack of discriminative power, where models didn't sufficiently distinguish between similar but distinct relations, and the hubness problem, where certain points in high-dimensional spaces tend to be near many other points, leading to false positives when using simple nearest-neighbor searches.

Methodology

The authors propose a model that learns highly discriminative embeddings for both sentences and relations. The architecture consists of three main components:

Dual Encoders

The model uses a BERT-based encoder for sentences (using entity markers to highlight the subject and object) and a Sentence-BERT encoder for relation descriptions (e.g., "the place where an entity was born").

Discriminative Learning Constraints

Instead of just mapping items to a shared space, the model uses specific losses to force intra-class compactness and inter-class separation:

  • Sentence Loss ($L_{sent}$): Combines a standard softmax classifier with a similarity-based loss to ensure sentences of the same relation are clustered tightly together while pushing different relations apart.
  • Relation Loss ($L_{rel}$): Explicitly maximizes the distance between different relation embeddings to prevent overlap in the semantic space.
The proposed model architecture featuring dual encoders and a self-adaptive comparator network.

Self-Adaptive Comparator Network ($F$)

Rather than using a fixed distance metric like Cosine similarity, the authors use a two-layer neural network to "judge" if a sentence-relation pair is consistent. This learned verification process is more robust against the hubness problem.

Experiments and Results

The model was evaluated on two widely used benchmarks: Wiki-ZSL and FewRel. The authors tested the model by withholding $m$ relations during training (where $m = 5, 10, or 15$) and requiring the model to predict them at test time.

The proposed method significantly outperformed existing baselines across all metrics (Precision, Recall, and F1-score). On Wiki-ZSL ($m=5$), the model achieved an F1-score of 82.19, a significant jump over ZS-BERT’s 72.97. On FewRel ($m=5$), the model reached an F1-score of 86.69, compared to ZS-BERT’s 79.44.

Visualizing Success

The impact of discriminative learning is most evident in the resulting embedding space. t-SNE visualizations clearly show that while previous models (ZS-BERT) produce overlapping "clouds" of data, this new approach produces distinct, well-separated clusters.

t-SNE visualization comparing ZS-BERT (left) with the proposed model (right), showing clearer cluster separation.

Conclusion

This paper demonstrates that intra-class compactness is crucial for zero-shot tasks. By replacing static distance metrics with a trained comparator network and enforcing discriminative constraints, the model achieves a new state-of-the-art in Zero-Shot Relation Extraction.

Summary updated

Abstract

Zero-shot relation extraction (ZSRE) aims to predict target relations that cannot be observed during training. While most previous studies have focused on fully supervised relation extraction and achieved considerably high performance, less effort has been made towards ZSRE. This study proposes a new model incorporating discriminative embedding learning for both sentences and semantic relations. In addition, a self-adaptive comparator network is used to judge whether the relationship between a sentence and a relation is consistent. Experimental results on two benchmark datasets showed that the proposed method significantly outperforms the state-of-the-art methods.
BibTeX
@misc{tran2022improving,
  author = {Van-Hien Tran and Hiroki Ouchi and Taro Watanabe and Yūji Matsumoto},
  title = {{Improving Discriminative Learning for Zero-Shot Relation Extraction}},
  year = {2022},
  doi = {10.18653/v1/2022.spanlp-1.1},
}
Figure