Extracting Semantic Relations Across Sentence Boundaries

19 concepts17 papers1 proposal

Traditional relation extraction models operate under a strict constraint: they assume that the two entities forming a relational triple (such as [Chemical] -> causes -> [Disease]) reside within the boundaries of a single sentence. While sentence-bound extraction simplifies model design, it creates a severe inter-sentence dependency gap.

In complex domains like biomedical research, material science, and planetary geology, valuable facts are rarely contained in isolated sentences. A narrative might introduce a material or target in one sentence, and detail its properties, quantities, or experimental conditions several sentences later. As noted by Jerry Junyang Cheung et al. in their work on the POLYIE: A Dataset of Information Extraction from Polymer Material Scientific Literature dataset, standard sentence-level extraction methods result in fragmented data records because they cannot link these disparate mentions.

When an information extraction system is restricted to single-sentence analysis, it suffers from a major drop in recall (Parisa Kordjamshidi et al., Machine Reading of Biological Texts - Bacteria-Biotope Extraction). To build comprehensive knowledge bases from scientific literature, models must possess mechanisms for cross-sentence reasoning (Jiaju Lin et al., ECNUICA at SemEval-2021 Task 11: Rule based Information Extraction Pipeline).

Over the years, the NLP community has shifted from local, sentence-bound parsing to document-level architectures. Understanding why these cross-sentence approaches work requires looking at three distinct paradigms: discourse tracking, document-level graphs, and global self-attention.

Bridging the Gap with Discourse and Coreference

Early attempts to extract cross-sentence relations focused on resolving the linguistic mechanisms that cause information to span across sentences in the first place: anaphora (pronouns referring back to an entity) and ellipsis (the omission of a word or phrase).

If a text reads, "We analyzed the rock Cumberland. It contains high amounts of magnetite," a sentence-bound model cannot link Cumberland to magnetite because the second sentence uses the pronoun "It". As demonstrated by Kiri Wagstaff et al. in the development of the Mars Target Encyclopedia: Rock and Soil Composition Extracted From the Literature, a substantial percentage of scientific claims are lost unless the system can resolve these cross-sentence references.

To solve this, researchers integrated explicit discourse-tracking frameworks into the extraction pipeline:

  • Anaphora Resolution as a Pre-requisite: Nozomi Kobayashi et al. showed that in opinion mining, identifying the referents of pronouns before determining if an opinion exists significantly improves extraction accuracy (Opinion Extraction Using a Learning-Based Anaphora Resolution Technique). By resolving what an attribute refers to across sentence boundaries, the model establishes a semantic bridge where direct syntactic links are missing.
  • Centering Theory: Toru Hirano et al. incorporated Centering Theory—a linguistic framework that tracks the local focus of a discourse—to handle complex grammatical omissions across sentences (Detecting Semantic Relations between Named Entities in Text Using Contextual Features). By tracking which entity remains the "focus" as the narrative moves from one sentence to the next, the model can infer relationships even when the subject is completely omitted in subsequent clauses.

Document Graphs: Shortening the Path Distance

While discourse theories helped resolve pronouns, they struggled to scale to complex technical documents where relationships are implicit and do not use explicit anaphora. Standard dependency parsers only map structural relationships within a single sentence, leaving a total disconnect at the period.

To allow information to flow across the entire document, researchers began constructing document-level graphs. The core intuition is simple: transform the linear sequence of text into a network where cross-sentence entities are physically brought closer together.

Sentence 1: [Entity A] is a novel polymer.
               │
       (Coreference Link)
               ▼
Sentence 2: [It] exhibits high [Property B].

In a document graph, nodes represent words or named entities, and edges represent relationships. Crucially, these graphs introduce non-local edges, such as coreference links (connecting "It" to "Entity A") or adjacent-sentence links.

Fenia Christopoulou et al. utilized this approach by training a Graph Convolutional Neural Network (GCNN) on document-level graphs (Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network). Because graph convolutions propagate information from a node to its neighbors, stacking multiple GCNN layers allows representation vectors for Entity A and Property B to interact, even if they were originally separated by multiple sentences.

Similarly, Wuti Xiong et al. combined dependency graphs with lexical chains—sequences of semantically related words that span across a text—to maintain connectivity across sentence boundaries (Bacteria Biotope Relation Extraction via Lexical Chains and Dependency Graphs). By providing these explicit semantic pathways, graph-based models bypass the rigid boundaries of sentence-level grammar, allowing structural information to flow globally (Nagesh C. Panyam et al., Exploiting graph kernels for high performance biomedical relation extraction).

Global Self-Attention and Mention-Pair Scoring

While document graphs are powerful, they rely heavily on the accuracy of upstream tools like coreference resolvers and dependency parsers. If the parser makes a mistake, the graph breaks. Modern architectures avoid this dependency by leveraging the global receptive field of the Transformer architecture.

Instead of processing text one sentence at a time, models can ingest multi-sentence "snippets" or entire abstracts simultaneously (Youngrok Jang et al., Context aware Named Entity Recognition and Relation Extraction with Domain-specific language model). Because self-attention calculates a pairwise affinity score between every token in the input sequence, the physical distance between tokens becomes irrelevant. An entity mention at the start of an abstract can directly attend to a property mention at the very end.

A prominent realization of this idea is the Bi-affine Relation Attention Network (BRAN), introduced by Emma Strubell et al. (Attending to All Mention Pairs for Full Abstract Biological Relation Extraction; Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction).

Input Abstract: [Tokens 1 ... N]
       │
       ▼ (Transformer Self-Attention)
Contextualized Representations for All Tokens
       │
       ▼ (Bi-affine Classifier)
Simultaneous Scoring of All Mention Pairs (Cross-Sentence & Intra-Sentence)

BRAN completely removes sentence boundaries from the equation. It uses a Transformer to learn contextualized representations of the text and then applies a bi-affine scoring layer to evaluate all potential entity mention pairs across the entire abstract simultaneously. This approach is highly efficient and eliminates the need for hand-crafted linguistic features or rigid graph construction.

To further refine this document-wide reasoning, Shengqiong Wu et al. proposed the Mention-based Reasoning Network (MRN), which explicitly separates and then reunites local and global reasoning (MRN: A Locally and Globally Mention-Based Reasoning Network for Document-Level Relation Extraction). MRN uses 2D convolutions to capture localized, intra-sentence context, while employing a co-attention mechanism to aggregate global, document-level evidence. This dual-pathway design ensures that the model does not lose track of sharp, local syntactic details while synthesizing a broader narrative across multiple paragraphs.

By treating relation extraction as a document-level problem—whether through explicit discourse tracking, structural graph routing, or global self-attention—modern architectures successfully capture the complex, distributed nature of information in real-world text.

Key Foundational Work

Detecting Semantic Relations between Named Entities in Text Using Contextual Features (2007) — contributed 1 concept elements to this direction.
Opinion Extraction Using a Learning-Based Anaphora Resolution Technique (2005) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_cross_sentence_relation_extraction_inter_8ba03d5a,
  title = {cross-sentence relation extraction (inter-sentential dependency)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-sentence-relation-extraction-inter-sentential-dependency/rdr_8ba03d5aa2d240ca9ba0806eb117a4e1},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_cross_sentence_relation_extraction_inter_8ba03d5a_v1,
  title = {cross-sentence relation extraction (inter-sentential dependency)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/cross-sentence-relation-extraction-inter-sentential-dependency/rdr_8ba03d5aa2d240ca9ba0806eb117a4e1/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake