Relational Reasoning Across Sentence Boundaries

231 concepts185 papers2 proposals

Traditional relation extraction (RE) models operate under a comfortable assumption: the evidence required to identify a relationship between two entities resides within the boundaries of a single sentence. While this holds true for simple facts, real-world knowledge is rarely so neatly packaged. As noted by Xu Han et al. in their overview of neural relation extraction toolkits like OpenNRE: An Open and Extensible Toolkit for Neural Relation Extraction (emnlp 2019), moving from sentence-level to document-level extraction introduces a massive leap in complexity. In long-form text, entities are mentioned repeatedly across different paragraphs, and identifying their relationships requires synthesizing information over long distances.

Why Sentence-Bound Models Fail

To understand why document-level relation extraction (DRE) requires a paradigm shift, consider how standard models process text. Sentence-level models rely heavily on local syntactic structures and immediate contextual proximity. However, as Johannes Deleu et al. demonstrate with the DWIE: an entity-centric dataset for multi-task document-level information extraction (arxiv 2020) dataset, a significant portion of semantic relations cannot be extracted locally. Instead, they require resolving coreferences and performing multi-hop reasoning across sentence boundaries.

When models are restricted to isolated sentences, they suffer from a severe drop in performance on real-world "hard cases" where relations are implicit rather than explicit—a challenge highlighted by Qiao Cheng in the creation of the HacRED: A Large-Scale Relation Extraction Dataset Toward Hard Cases in Practical Applications (acl 2021) benchmark. For instance, a document might introduce a company in paragraph one, describe a technology in paragraph three, and only reveal that the company developed the technology by connecting clues scattered across the entire text.

Graph-Based Modeling: Bridging the Structural Gap

One of the most intuitive solutions to this problem is to transform the linear sequence of a document into a non-linear graph structure. Instead of forcing a Transformer to attend to every token across a massive context window—which is computationally expensive and dilutes local signals—graph-based models construct explicit networks of mentions, entities, and sentences.

Fenia Christopoulou pioneered this shift with an edge-oriented graph approach in Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs (emnlp 2019), focusing on iteratively refining edge representations to pass information between distant entity mentions. Building on this intuition, Hieu Minh Tran et al. introduced EoGANE in The Dots Have Their Values: Exploiting the Node-Edge Connections in Graph-based Neural Models for Document-level Relation Extraction (emnlp 2020), which enforces consistency between nodes (entities) and edges (relations) to ensure that multiple mentions of the same entity maintain semantic coherence across paragraphs.

To balance local details with global document structure, Damai Dai et al. proposed a coarse-to-fine representation framework in Coarse-to-Fine Entity Representations for Document-Level Relation Extraction (NLPCC (2) 2023), combining global document context with fine-grained path information between specific entities.

Disentangling Local and Global Context

A key insight in document-level reasoning is that intra-sentence relations (within the same sentence) and inter-sentence relations (across different sentences) require fundamentally different types of cognitive processing.

[Sentence 1: Alice works at Acme Corp.] ---> Local Syntax (Intra-sentential)
       |
       | (Coreference / Multi-hop Bridge)
       v
[Sentence 2: The company is based in Paris.] ---> Global Discourse (Inter-sentential)

To address this distinction, Shuang Zeng et al. developed SIRE in SIRE: Separate Intra- and Inter-sentential Reasoning for Document-level Relation Extraction (acl 2021), an architecture that explicitly separates intra- and inter-sentential reasoning tracks. By isolating local syntactic patterns from global logical reasoning, the model avoids getting overwhelmed by background text. Similarly, Shengqiong Wu et al. proposed MRN in MRN: A Locally and Globally Mention-Based Reasoning Network for Document-Level Relation Extraction (acl 2021), which utilizes 2D convolutions to capture local mention context alongside a co-attention mechanism designed specifically for global, document-wide dependencies.

Alternative Formulations: QA, Summarization, and Generation

Beyond graph structures, researchers have reframed DRE by leveraging the pre-trained capabilities of large language models through alternative task formulations:

Discourse-Aware Pre-training and Denoising

Standard language model pre-training objectives (like masked language modeling) are highly effective for local syntax but fail to capture how sentences relate to one another over long distances. To address this, Dan Iter et al. introduced CONPONO in Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models (acl 2020), a contrastive objective that explicitly trains models to track discourse structures across multi-sentence spans.

Furthermore, when training data is generated automatically via distant supervision, document-level context introduces massive label noise because not every mention of an entity pair expresses the underlying relation. Chaojun Xiao et al. addressed this in Denoising Relation Extraction from Document-level Distant Supervision (emnlp 2020) by designing specialized pre-training tasks that align facts and match mentions, effectively filtering out irrelevant entity pairs across long-form text.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cross_sentence_relation_extraction_docum_d0a61a21,
  title = {cross-sentence relation extraction (document-level relation extraction)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-sentence-relation-extraction-document-level-relation-extraction/rdr_d0a61a2114a84992957e6fdf202ef264},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_cross_sentence_relation_extraction_docum_d0a61a21_v1,
  title = {cross-sentence relation extraction (document-level relation extraction)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-sentence-relation-extraction-document-level-relation-extraction/rdr_d0a61a2114a84992957e6fdf202ef264/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake