TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations

Xianming Li, Xiaotian Luo, Chenghao Dong, Daichuan Yang, Beidi Luan, Zhen HeEMNLP 2021 · OpenReview

Abstract

Joint extraction of entities and relations from unstructured texts to form factual triples is a fundamental task of constructing a Knowledge Base (KB). A common method is to decode triples by predicting entity pairs to obtain the corresponding relation. However, it is still challenging to handle this task efficiently, especially for the overlapping triple problem. To address such a problem, this paper proposes a novel efficient entities and relations extraction model called TDEER, which stands for Translating Decoding Schema for Joint Extraction of Entities and Relations. Unlike the common approaches, the proposed translating decoding schema regards the relation as a translating operation from subject to objects, i.e., TDEER decodes triples as subject + relation → objects. TDEER can naturally handle the overlapping triple problem, because the translating decoding schema can recognize all possible triples, including overlapping and non-overlapping triples. To enhance model robustness, we introduce negative samples to alleviate error accumulation at different stages. Extensive experiments on public datasets demonstrate that TDEER produces competitive results compared with the state-of-the-art (SOTA) baselines. Furthermore, the computation complexity analysis indicates that TDEER is more efficient than powerful baselines. Especially, the proposed TDEER is 2 times faster than the recent SOTA models.

Introduction

Extracting factual triples—structured (subject, relation, object) sets—from unstructured text is the backbone of Knowledge Base (KB) construction. However, two major hurdles remain in current joint extraction methods: the overlapping triple problem and computational efficiency.

The Overlapping Triple Problem occurs when entities participate in multiple relations within a single sentence. This manifests in two ways: Single Entity Overlap (SEO), where one entity belongs to different triples, and Entity Pair Overlap (EPO), where the same two entities share multiple relations.

Examples of Normal, EPO, and SEO overlapping patterns in triple extraction.

Furthermore, recent high-performing models often suffer from high computational complexity ($O(n^2)$), making them slow for large-scale deployment. TDEER aims to solve both by rethinking the decoding process.

Methodology

TDEER (Translating Decoding Schema for Joint Extraction of Entities and Relations) treats relations as "translating" operations from subjects to objects ($subject + relation \to objects$). The framework utilizes a three-stage pipeline:

  1. Entity Tagging: A span-based tagger identifies all potential subjects and objects.
  2. Relation Detection: A multi-label classifier identifies relations present in the sentence.
  3. Translating Decoding: For every pair of a detected subject and relation, the model "translates" them into a query. It combines the subject representation ($v_{sub}$) and relation representation ($v_{rel}$) via addition ($u = v_{sub} + v_{rel}$) and uses an attention mechanism to pinpoint the object's start position.
The TDEER architecture, illustrating the flow from BERT encoding to multi-label classification and translating decoding.

To prevent "cascading errors" common in multi-stage models, TDEER introduces a negative sampling strategy. During training, the model is fed incorrect subject-relation pairs and taught to predict "no object," making the final decoder more robust to noise.

Key Insights

  • Relation as Translation: Borrowing from Knowledge Graph embeddings (like TransE), TDEER treats the relation as a specific vector offset in semantic space.
  • Search Space Reduction: TDEER only iterates through detected subjects and relations, reducing complexity to $O(n + sr)$, where $s$ and $r$ are typically much smaller than the sequence length $n$.
  • Attention-Guided Decoding: The combined subject-relation vector queries the sentence via attention, allowing the model to dynamically focus on relevant parts for different relations.

Experiments and Results

The authors evaluated TDEER on three benchmarks: NYT (large, high overlap), WebNLG (high relation density), and NYT11-HRL (focus on normal triples).

Key Results

  • Superior Accuracy: TDEER-BERT achieved F1-scores of 92.5% on NYT and 93.1% on WebNLG, matching or exceeding previous SOTA models like TPLinker.
  • Significant Speedup: TDEER is roughly 2x faster than TPLinker. On NYT, TDEER's inference time was 31.4ms per sample vs. TPLinker's 82.7ms.
  • Complexity Robustness: For sentences with five or more triples, TDEER outperformed TPLinker by 2.8% on the NYT dataset.

Conclusion

TDEER provides a novel, efficient framework for joint entity and relation extraction. By framing relations as translations and employing a robust decoding schema, it effectively handles overlapping triples while maintaining high inference speeds. While the negative sampling strategy increases training overhead, the significant gains in inference efficiency and accuracy make it a powerful tool for large-scale Knowledge Base construction. Future work will explore more efficient ways to handle error accumulation and apply this schema to broader information extraction tasks.

Summary updated

Abstract

Joint extraction of entities and relations from unstructured texts to form factual triples is a fundamental task of constructing a Knowledge Base (KB). A common method is to decode triples by predicting entity pairs to obtain the corresponding relation. However, it is still challenging to handle this task efficiently, especially for the overlapping triple problem. To address such a problem, this paper proposes a novel efficient entities and relations extraction model called TDEER, which stands for Translating Decoding Schema for Joint Extraction of Entities and Relations. Unlike the common approaches, the proposed translating decoding schema regards the relation as a translating operation from subject to objects, i.e., TDEER decodes triples as subject + relation → objects. TDEER can naturally handle the overlapping triple problem, because the translating decoding schema can recognize all possible triples, including overlapping and non-overlapping triples. To enhance model robustness, we introduce negative samples to alleviate error accumulation at different stages. Extensive experiments on public datasets demonstrate that TDEER produces competitive results compared with the state-of-the-art (SOTA) baselines. Furthermore, the computation complexity analysis indicates that TDEER is more efficient than powerful baselines. Especially, the proposed TDEER is 2 times faster than the recent SOTA models. The code is available at https://github.com/4AI/TDEER.
BibTeX
@inproceedings{li2021tdeer,
  author = {Xianming Li and Xiaotian Luo and Chenghao Dong and Daichuan Yang and Beidi Luan and Zhen He},
  title = {{TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2021},
  url = {https://openreview.net/forum?id=nDxb1S6boO},
}
Figure