Document-level Relation Extraction with Relation Correlations

Ridong Han, Tao Peng, Benyou Wang, Lu Liu, Xiang Wan

TL;DR

Document-level relation extraction models often struggle to predict rare relations or handle entity pairs that express multiple relationships at once. This paper introduces Relation Correlations—the concept that real-world relationships naturally co-occur—using two novel co-occurrence prediction sub-tasks to share knowledge between data-rich and data-scarce relations. This approach significantly boosts performance, yielding up to a 12.39% F1 improvement on complex multi-label extractions.

What Problem Does This Paper Address?

Document-level Relation Extraction (DocRE) requires a model to identify relationships between all entity pairs across an entire document. Unlike sentence-level extraction, DocRE is highly realistic—over 40% of relational facts in standard datasets can only be extracted by connecting clues across multiple sentences.

However, current DocRE models hit a wall when facing two major, overlooked challenges:

  1. The Long-Tail Problem: A few relations have thousands of training examples, but the vast majority have very few (e.g., less than 200). Standard training causes models to severely underfit these rare relations.
  2. The Multi-Label Problem: Around 7% of entity pairs express multiple relations simultaneously (e.g., a person's country of citizenship and their place of birth). Classifying these requires drawing highly complex, overlapping decision boundaries.
Figure 2: Left figure shows the results of BERTBASE on long-tailed relations; right figure is the results on multi- label entity pairs. We run 5 times with different seeds, and show the mean and standard deviation bar.

Existing works focus almost entirely on building better contextual representations of text, treating relation labels as completely independent. This paper argues that relations are inherently interdependent, and modeling their correlations can solve both bottlenecks.

Methods

The authors propose a model-agnostic framework that learns Relation Correlations through two auxiliary sub-tasks, using relation embeddings as a bridge:

  • Joint Embedding Learning (JE): Instead of just passing the document text to a Pre-trained Language Model (like BERT), the authors append a sequence of all possible relation labels to the text input: [CLS] document tokens [SEP] relation_1, relation_2... [SEP]. This forces the model to map text and labels into the exact same hidden space, allowing self-attention layers to automatically capture dependencies between them.
  • Coarse-grained Relation Co-occurrence Prediction (CRCP): This label-level sub-task trains the model to predict whether a specific target relation is likely to appear given a set of other expressed relations in the document, independent of the specific entities.
  • Fine-grained Relation Co-occurrence Prediction (FRCP): Because relationships ultimately depend on specific entity pairs, this sub-task introduces a Relation-Aggregated Embedding (RAE). It dynamically weights and aggregates relation embeddings based on token-to-relation attention scores to predict co-occurrences conditioned on both the text context and the entity pair.

During final inference, these correlation-aware relation embeddings are fused back into the entity pair features to guide the final relation classifier.

Datasets

The model was evaluated on two widely recognized document-level relation extraction benchmarks:

  • DocRED: A large-scale, crowdsourced dataset built from Wikipedia articles containing 96 relation types.
  • DWIE: An entity-centric dataset for multi-task document-level information extraction containing 65 relation types.

Key Insights and Ideas

  • Knowledge Transfer via Anchors: Data-rich "head" relations can act as anchors to support data-scarce "tail" relations. If the model learns that date of birth strongly correlates with place of birth, it can leverage the abundance of the former to improve predictions for the latter when training data is scarce.
  • Semantic Distance as a Delineator: For multi-label pairs, co-occurrence frequency acts as a proxy for semantic distance. Knowing which relations frequently overlap helps the classifier narrow down its search space and draw sharper decision boundaries.

Key Results

The proposed model, BERT-Correl<sub>BASE</sub>, consistently outperformed strong baselines (including ATLOP and GAIN) across both datasets:

  • Mitigating the Long-Tail: On data-scarce relations with fewer than 100 training instances, the model improved the macro-F1 score by up to 6.49% on DocRED and 6.12% on DWIE.
  • Conquering Multi-Labels: For entity pairs expressing four distinct relations simultaneously, the model achieved a massive 12.39% F1 improvement over the baseline.
  • Captured vs. Statistical Correlations: Visualizations of the learned embedding similarities closely mirrored the actual statistical co-occurrence matrices of the datasets, proving that the auxiliary tasks successfully captured real-world semantic links.
Figure 3: Relation correlations on DocRED. For bet- ter visual effect, we only show part of relations. See Appendix B for the complete figure.

Limitations and Future Work

The authors do not explicitly detail limitations in the text, but a few architectural constraints are apparent:

  • Sequence Length Overhead: Appending all relation labels to the input text sequence increases the token count. For datasets with hundreds of relation types, this could easily exceed the standard 512-token limit of models like BERT.
  • Noise in Learned Matrices: While the learned correlation matrices closely match statistical realities, they still contain minor noise points that could occasionally misguide the classifier. Future work could explore filtering or regularizing these correlation matrices.

Summary updated

Abstract

Document-level relation extraction faces two overlooked challenges: long-tail problem and multi-label problem. Previous work focuses mainly on obtaining better contextual representations for entity pairs, hardly address the above challenges. In this paper, we analyze the co-occurrence correlation of relations, and introduce it into DocRE task for the first time. We argue that the correlations can not only transfer knowledge between data-rich relations and data-scarce ones to assist in the training of tailed relations, but also reflect semantic distance guiding the classifier to identify semantically close relations for multi-label entity pairs. Specifically, we use relation embedding as a medium, and propose two co-occurrence prediction sub-tasks from both coarse- and fine-grained perspectives to capture relation correlations. Finally, the learned correlation-aware embeddings are used to guide the extraction of relational facts. Substantial experiments on two popular DocRE datasets are conducted, and our method achieves superior results compared to baselines. Insightful analysis also demonstrates the potential of relation correlations to address the above challenges.
BibTeX
@inproceedings{han2022document,
  author = {Ridong Han and Tao Peng and Benyou Wang and Lu Liu and Xiang Wan},
  title = {{Document-level Relation Extraction with Relation Correlations}},
  booktitle = {Neural Networks},
  year = {2022},
}
Figure