Reasoning with Latent Structure Refinement for Document-Level Relation Extraction

Guoshun Nan, Zhijiang Guo, Ivan Sekulic, Wei Lu

Abstract

Document-level relation extraction requires integrating information within and across multiple sentences of a document and capturing complex interactions between inter-sentence entities. However, effective aggregation of relevant information in the document remains a challenging research question. Existing approaches construct static document-level graphs based on syntactic trees, co-references or heuristics from the unstructured text to model the dependencies. Unlike previous methods that may not be able to capture rich non-local interactions for inference, we propose a novel model that empowers the relational reasoning across sentences by automatically inducing the latent document-level graph. We further develop a refinement strategy, which enables the model to incrementally aggregate relevant information for multi-hop reasoning. Specifically, our model achieves an F1 score of 59.05 on a large-scale document-level dataset (DocRED), significantly improving over the previous results, and also yields new state-of-the-art results on the CDR and GDA dataset. Furthermore, extensive analyses show that the model is able to discover more accurate inter-sentence relations.

Introduction

Relation Extraction (RE) is a fundamental task in information extraction, traditionally focused on identifying relationships between entities within a single sentence. However, real-world documents often distribute relational information across multiple sentences, requiring a system to synthesize facts from disparate parts of the text. For instance, identifying a relationship between Entity A and Entity C might require finding a "bridge" through Entity B mentioned elsewhere in the document.

Existing document-level RE models typically rely on static graph structures—such as coreference links or syntactic dependency trees—to model these long-range dependencies. These static structures are often noisy, incomplete, or misaligned with the specific reasoning required for the extraction task. This paper introduces Latent Structure Refinement (LSR), a model that moves beyond rigid, pre-defined graphs by automatically inducing and iteratively refining a latent document-level graph tailored for relational reasoning.

Methods

The LSR model architecture is divided into three primary stages: node construction, dynamic reasoning, and classification.

Node Constructor

The model first identifies three distinct types of nodes to represent the document's content:

  • Mention Nodes: Represent individual occurrences of entities in the text.
  • Entity Nodes: Aggregate information from all mentions of a specific entity (typically via averaging) to form a global representation.
  • Meta Dependency Path (MDP) Nodes: Tokens located on the shortest syntactic path between mentions within a sentence, providing essential local context.

Dynamic Reasoner

The core innovation of LSR lies in the Dynamic Reasoner, which avoids the limitations of static graphs through two key mechanisms:

  1. Structure Induction: Using a variant of the Matrix-Tree Theorem, the model calculates the probability of edges between any two nodes, effectively "learning" the most relevant connections for the task.
  2. Iterative Refinement: The model employs Densely Connected Graph Convolutional Networks (GCNs) to propagate information. Crucially, this is an iterative process where node representations are updated and then used to induce a more refined graph structure. This allows the model to capture increasingly complex, multi-hop interactions.
The Dynamic Reasoner architecture showing structure induction and iterative refinement.

Classifier

Finally, the refined entity representations are passed to a bilinear classifier to predict the probability of various relationship types between entity pairs.

Results and Analysis

The authors evaluated LSR on three major benchmarks: DocRED (large-scale human-annotated), CDR (biomedical chemical-disease), and GDA (biomedical gene-disease).

Performance on DocRED

LSR achieved a state-of-the-art F1 score of 59.05 on the DocRED dataset when using BERT as the encoder backbone. Ablation studies revealed that the "Structure Induction" module is critical, as its removal led to a 3.26-point drop in F1.

Inter-sentence Reasoning

A key finding is LSR's superior performance in extracting inter-sentence relations—those where entities appear in different sentences. This confirms that the model successfully performs document-wide synthesis rather than relying on local patterns.

Performance comparison of LSR against baselines across different F1 metrics and refinement blocks.

As shown in the performance analysis, the iterative refinement strategy is vital. The model typically reaches peak performance at 2 refinement blocks, particularly for inter-sentence relations. Beyond this point, performance may slightly decline due to overfitting on the training data.

Biomedical Benchmarks

On the GDA dataset, a simplified version of LSR (omitting MDP nodes) achieved an F1 of 82.2, outperforming previous models that relied on complex, hand-crafted heuristics.

Conclusion

The LSR model demonstrates that inducing a latent graph structure is more effective for document-level relation extraction than relying on static, externally-derived graphs. By allowing the model to learn and refine its own reasoning paths, LSR achieves significant gains in identifying complex, non-local interactions. Future work may focus on automating the induction of internal sentence structures (MDP nodes) to further reduce reliance on external linguistic tools and improving the efficiency of latent graph induction for extremely long documents.

Summary updated

Abstract

Document-level relation extraction requires integrating information within and across multiple sentences of a document and capturing complex interactions between inter-sentence entities. However, effective aggregation of relevant information in the document remains a challenging research question. Existing approaches construct static document-level graphs based on syntactic trees, co-references or heuristics from the unstructured text to model the dependencies. Unlike previous methods that may not be able to capture rich non-local interactions for inference, we propose a novel model that empowers the relational reasoning across sentences by automatically inducing the latent document-level graph. We further develop a refinement strategy, which enables the model to incrementally aggregate relevant information for multi-hop reasoning. Specifically, our model achieves an F1 score of 59.05 on a large-scale document-level dataset (DocRED), significantly improving over the previous results, and also yields new state-of-the-art results on the CDR and GDA dataset. Furthermore, extensive analyses show that the model is able to discover more accurate inter-sentence relations.
BibTeX
@inproceedings{nan2020reasoning,
  author = {Guoshun Nan and Zhijiang Guo and Ivan Sekulic and Wei Lu},
  title = {{Reasoning with Latent Structure Refinement for Document-Level Relation Extraction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  url = {https://openreview.net/forum?id=BWeCjrGW4Q2},
}
Figure