MORE: A Metric Learning Based Framework for Open-domain Relation Extraction

Yutong Wang, Renze Lou, Kai Zhang, Mao Yan Chen, Yujiu Yang

Abstract

Open relation extraction (OpenRE) is the task of extracting relation schemes from open-domain corpora. Most existing OpenRE methods either do not fully benefit from high-quality labeled corpora or can not learn semantic representation directly, affecting downstream clustering efficiency. To address these problems, we propose a novel learning framework named MORE (Metric learning-based Open Relation Extraction). The framework utilizes deep metric learning to obtain rich supervision signals from labeled data and drive the neural model to learn semantic relational representation directly. Experiments result in two real-world datasets show that our method outperforms other state-of-the-art baselines.

Introduction

In Natural Language Processing, Relation Extraction (RE) identifies connections between entities (e.g., "London" is the "capital of" the "UK"). While traditional RE relies on fixed labels, Open Relation Extraction (OpenRE) seeks to discover relations in open-domain text without a predefined schema. Current methods often struggle with surface-level patterns or inefficient similarity classifiers. The MORE framework addresses this by mapping sentences into a high-dimensional space where semantic relationships are directly represented, facilitating more accurate clustering.

The MORE Framework

The MORE architecture consists of three primary components designed to produce robust relational representations.

Overview of the MORE framework architecture showing the flow from training samples to supervision signals.

1. Neural Encoders

MORE is model-agnostic, supporting various encoders like CNNs or pre-trained BERT. For BERT, entity-marking techniques are employed to focus the model's attention on the specific relationship between entity pairs.

2. Ranked List Loss (RLL)

Unlike standard Triplet Loss, Ranked List Loss considers the entire data batch. It pulls all positive instances toward an anchor while pushing negative instances beyond a defined margin. By weighting "hard" negatives more heavily, RLL provides a richer supervision signal, creating a more discriminative feature space.

3. Virtual Adversarial Training (VAT)

To handle the noise inherent in open-domain text, VAT introduces small, "worst-case" perturbations to word embeddings. The model is trained to remain consistent despite these changes, effectively "smoothing" the semantic space and increasing robustness against diverse linguistic expressions.

Experimental Results

The framework was evaluated on the FewRel and NYT+FB-sup benchmarks. MORE demonstrated significant performance gains, particularly when using BERT as the encoder.

Performance comparison on the FewRel dataset, showing MORE outperforming state-of-the-art baselines.

On the FewRel dataset, MORE (BERT) achieved an F1 score of 74.5, a substantial improvement over the previous state-of-the-art (RSNs) at 59.9. Even in noisy environments like NYT+FB-sup, MORE maintained a clear lead with a 49.7 F1 score. Ablation studies confirmed that VAT provides additional gains in robustness, particularly for CNN-based implementations, by smoothing the probability distribution space.

Conclusion

The MORE framework shifts the paradigm of OpenRE from similarity classification to direct semantic representation learning. By leveraging Ranked List Loss and Virtual Adversarial Training, MORE creates a well-structured feature space that significantly improves the discovery of unseen relations. While the choice of cluster numbers remains a challenge, the improved representations provide a much stronger foundation for open-domain knowledge extraction.

Summary updated

Abstract

Open relation extraction (OpenRE) is the task of extracting relation schemes from open-domain corpora. Most existing OpenRE methods either do not fully benefit from high-quality labeled corpora or can not learn semantic representation directly, affecting downstream clustering efficiency. To address these problems, in this work, we propose a novel learning framework named MORE (Metric learning-based Open Relation Extraction). The framework utilizes deep metric learning to obtain rich supervision signals from labeled data and drive the neural model to learn semantic relational representation directly. Experiments result in two real-world datasets show that our method outperforms other state-of-the-art baselines. Our source code is available on Github.
BibTeX
@inproceedings{wang2021more,
  author = {Yutong Wang and Renze Lou and Kai Zhang and Mao Yan Chen and Yujiu Yang},
  title = {{MORE: A Metric Learning Based Framework for Open-domain Relation Extraction}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2021},
  doi = {10.1109/icassp39728.2021.9413437},
}
Figure