A Conditional Cascade Model for Relational Triple Extraction

Feiliang Ren, Longhui Zhang, Shujuan Yin, Xiaofeng Zhao, Shilei Liu, Bochao Li

Abstract

Relational Triple Extraction (RTE) is a fundamental task in information extraction, yet it suffers from a significant class imbalance between non-entity tokens and actual relational triples. We present ConCasRTE, a conditional cascade framework that decomposes the extraction process into three distinct steps: subject tagging, object tagging, and relation classification. By incorporating a novel confidence threshold-based loss function that ignores "easy" negative samples, ConCasRTE focuses training on harder, more informative data. Experimental results on NYT and WebNLG benchmarks demonstrate that ConCasRTE achieves state-of-the-art performance, particularly in complex scenarios involving overlapping triples.

Introduction

Relational Triple Extraction (RTE) aims to identify structured facts from unstructured text in the form of (subject, relation, object) triples. For instance, from the sentence "Arhus Airport is located in Tirstrup, Denmark," a model should extract (Arhus Airport, is located in, Tirstrup) and (Arhus Airport, is located in, Denmark).

A major challenge in RTE is the extreme class imbalance. In a typical sentence, most tokens do not belong to any entity, and most entity pairs do not share a relation. Standard tagging-based models often struggle with this, becoming biased toward the majority "negative" class (no relation), which leads to poor recall for actual triples.

The ConCasRTE Framework

ConCasRTE addresses these issues through a three-step cascade architecture, refining the typical two-step approach used in previous state-of-the-art models like CasRel.

The ConCasRTE architecture illustrating the three-step cascade: Subject-Tagger, Object-Tagger, and Relation Extraction (RE).

The process is structured as follows:

  1. Subject-Tagger: The model first identifies the start and end positions of all potential subjects in the sentence.
  2. Object-Tagger: For each identified subject, the model identifies potential objects. This stage uses a Hadamard product to combine subject-specific features with token representations, creating a strong conditional dependency.
  3. Relation Extraction (RE): Finally, for each (subject, object) pair, the model predicts the specific relation.

By separating object and relation extraction, the model reduces the candidate space at each step, mitigating the impact of class imbalance.

Confidence Threshold-based Loss

The core innovation of ConCasRTE is its "Confidence Threshold-based Loss." Traditional cross-entropy loss penalizes all incorrect predictions equally. However, the authors observe that many negative samples are "easy" for the model to learn. Once the model's confidence in a correct prediction exceeds a certain threshold ($C$), the loss for that sample is set to zero. This prevents the gradients from being dominated by a massive number of easy negative samples, allowing the model to focus on the "hard" positive and negative samples that are crucial for high-precision extraction.

Experimental Results

The model was evaluated on the NYT and WebNLG datasets, which are standard benchmarks for RTE. ConCasRTE achieved new state-of-the-art F1-scores of 92.6% on NYT and 93.1% on WebNLG.

Performance on Complex Scenarios

One of the most significant strengths of ConCasRTE is its ability to handle complex sentences, such as those with Single Entity Overlap (SEO) or Entity Pair Overlap (EPO).

F1-score comparison on different sentence categories and triple counts (T) for NYT* and WebNLG* datasets.

As shown in the table above, ConCasRTE consistently outperforms baseline models like CasRel and TPLinker across all categories. Notably, on NYT* sentences containing five or more triples ($T \ge 5$), ConCasRTE maintains an F1-score of 91.7%, whereas CasRel's performance drops significantly to 83.7%. This demonstrates the robustness of the conditional cascade approach in high-density relational environments.

Conclusion

ConCasRTE introduces a powerful combination of a three-step cascade framework and a targeted loss function to solve the class imbalance problem in Relational Triple Extraction. By focusing training on difficult samples and conditioning object/relation extraction on identified subjects, the model sets a new standard for performance on standard benchmarks and complex real-world data.

Summary updated

BibTeX
@inproceedings{ren2021conditional,
  author = {Feiliang Ren and Longhui Zhang and Shujuan Yin and Xiaofeng Zhao and Shilei Liu and Bochao Li},
  title = {{A Conditional Cascade Model for Relational Triple Extraction}},
  booktitle = {ACM International Conference on Information and Knowledge Management},
  year = {2021},
  url = {https://openreview.net/forum?id=JJU9t9_aJTS},
}
Figure