Element Intervention for Open Relation Extraction

Fangchao Liu, Lingyong Yan, Hongyu Lin, Xianpei Han, Le Sun

Introduction

Open Relation Extraction (OpenRE) is a critical task in natural language processing that involves identifying and clustering semantic relationships between entities without a predefined set of labels. While distant supervision has been widely used to generate training data for this task, it introduces a significant challenge: noise. Models trained on distantly supervised data often learn "spurious correlations"—shortcuts where they rely on specific entity pairs or common phrases rather than the actual underlying semantic relationship. This phenomenon can lead to "model collapse," where the system fails to generalize to new, unseen examples.

Methodology: Causal Intervention

To address these issues, this research introduces a novel framework based on Causal Inference. The authors propose a Structural Causal Model (SCM) to identify how entities and contexts act as "confounders" that mislead the model. By identifying the "backdoor path" where the relation prototype influences both the entities and the context, they can apply the "do-operation" to break these spurious links.

The Structural Causal Model (a) and the proposed interventions on entities (b) and context (c).

The implementation, termed EI-ORE (Element Intervention for OpenRE), consists of two primary modules:

  1. Hierarchy-based Entity Ranking (Hyber): This module intervenes on entities by fixing the context and sampling alternative entities from a knowledge graph hierarchy (e.g., WikiData). By ranking these entities as Sibling, Cousin, or Other, the model learns that the relation should remain consistent even when specific entities are substituted with similar ones.
  2. Generation-based Context Contrasting (Gcc): This module intervenes on the context using a pre-trained language model (T5) to generate variations of the sentence. It creates positive samples (renaming the relation) and negative samples (replacing the relation), forcing the model to focus on the semantic signals that truly define the relationship.

Experimental Results

The authors evaluated EI-ORE on two versions of the T-REx dataset: the cleaner T-REx SPO and the larger, noisier T-REx DS, which contains 12 million sentences generated via distant supervision.

Comprehensive performance comparison of EI-ORE against state-of-the-art models on T-REx SPO and T-REx DS datasets.

As shown in the results table, EI-ORE significantly outperforms previous state-of-the-art models such as SelfORE and UIE. On the cleaner T-REx SPO dataset, it improved the $B^3$ F1 score by 3.9%. More impressively, on the noisy T-REx DS dataset, the performance boost exceeded 10%. These results demonstrate that causal intervention effectively stabilizes the model against the noise inherent in distantly supervised data.

Conclusion and Future Work

By reframing OpenRE through the lens of causal inference, this work provides a principled method for achieving robustness in information extraction. While the approach currently relies on external tools like WikiData and T5, which may limit its application in highly specialized domains, the stability and performance gains are substantial. Future research may explore extending this causal framework to other noisy information extraction tasks.

Summary updated

Abstract

Open relation extraction aims to cluster relation instances referring to the same underlying relation, which is a critical step for general relation extraction. Current OpenRE models are commonly trained on the datasets generated from distant supervision, which often results in instability and makes the model easily collapsed. In this paper, we revisit the procedure of OpenRE from a causal view. By formulating OpenRE using a structural causal model, we identify that the above-mentioned problems stem from the spurious correlations from entities and context to the relation type. To address this issue, we conduct \emph{Element Intervention}, which intervenes on the context and entities respectively to obtain the underlying causal effects of them. We also provide two specific implementations of the interventions based on entity ranking and context contrasting. Experimental results on unsupervised relation extraction datasets show that our methods outperform previous state-of-the-art methods and are robust across different datasets.
BibTeX
@inproceedings{liu2021element,
  author = {Fangchao Liu and Lingyong Yan and Hongyu Lin and Xianpei Han and Le Sun},
  title = {{Element Intervention for Open Relation Extraction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.361},
}
Figure