Traditional Information Extraction (IE) operates on a simple, often unspoken assumption: if two entities are related, they will appear in the same sentence. This "sentence boundary" has historically acted as a hard constraint for models, which look for linguistic triggers or dependency paths within a single string of words to identify facts.
However, human knowledge is rarely so neatly packaged. In scientific literature, medical reports, and news, critical information is frequently fragmented. An entity might be introduced in the first sentence, while its relationship to a second entity is only clarified three sentences later through a pronoun or a shared context. Research by Hoifung Poon and Chris Quirk in Distant Supervision for Relation Extraction beyond the Sentence Boundary highlights that restricting extraction to single sentences leads to a systematic loss of complex biological events and interactions.
The Locality Constraint
The primary bottleneck in standard Relation Extraction (RE) is the locality constraint. Most models, such as those discussed by Liyuan Liu and Xiang Ren in Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach, rely on the co-occurrence of two entities within a single sentence to trigger a classification.
When we break this constraint, the search space for relations grows exponentially. If a document has $N$ sentences, a model must now consider entity pairs across all $N$ sentences, most of which are unrelated. Furthermore, the linguistic "bridge" between these entities is no longer a simple grammatical dependency; it is a discourse-level connection that requires understanding coreference and narrative flow.
Bridging the Gap with Document Graphs
To move beyond the sentence boundary, researchers have shifted from viewing text as a linear sequence to viewing it as a document graph. In this framing, nodes represent entities or mentions, and edges represent various types of relationships:
- Intra-sentence edges: Standard syntactic dependencies.
- Inter-sentence edges: Coreference links (e.g., "he" referring back to "Dr. Smith") or adjacent sentence connections.
By processing these graphs with specialized architectures like Graph LSTMs, as demonstrated by Hoifung Poon and colleagues in Cross-Sentence N-ary Relation Extraction with Graph LSTMs, models can "flow" information across the document. This allows the system to recognize that a drug mentioned in paragraph one interacts with a gene mentioned in paragraph two because they are linked through a chain of mentions and biological context.
Multi-Hop Reasoning and Inference Chains
A key intuition for why cross-sentence extraction works is the concept of inference chains. Sometimes, the relationship between Entity A and Entity C is never explicitly stated, but it can be inferred because A relates to B, and B relates to C.
Maosong Sun and Zhiyuan Liu explored this in Incorporating Relation Paths in Neural Relation Extraction. By building paths through intermediate entities, their model could learn from sentences that only mentioned one of the target entities. This multi-hop reasoning effectively doubles the recall of traditional systems by capturing the "long tail" of facts that are distributed across a narrative rather than condensed into a single clause.
Joint Extraction and Coreference
The most successful approaches to document-level extraction treat the problem holistically. Instead of a pipeline where entities are found first and relations second, models like SCIIE, developed by Yi Luan and Hannaneh Hajishirzi in Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction, perform multi-task learning.
By simultaneously identifying entities, relations, and coreference clusters, the model uses each task to reinforce the others. For example, knowing that "this protein" refers to "p53" (coreference) makes it much easier to identify the protein's function described in a later sentence (relation extraction). This synergy is essential for parsing complex scientific narratives where referential language is the norm.
Scaling to Holistic Representations
Recent work has pushed this even further by attempting to represent the entire meaning of a document in a single structure. Tahira Naseem and Jeffrey Flanigan introduced DocAMR, a framework for multi-sentence Abstract Meaning Representation. By merging sentence-level graphs into a unified document-level graph, they provide a standardized way to evaluate how well models capture relationships that span across an entire text.
As Hwee Tou Ng and Tapas Nayak argue in Effective Modeling of Encoder-Decoder Architecture for Joint Entity and Relation Extraction, the future of information extraction lies in breaking the "single-sentence constraint." By treating documents as interconnected webs of information rather than isolated strings of text, we can finally build knowledge bases that reflect the full complexity of human discourse.