Geneticists learned the hard way that the gene sitting next to a disease variant on the DNA strand is usually not the gene it controls. DNA folds in three dimensions, so a variant can touch a promoter a megabase away while skipping the genes in between. Language models reading long scientific papers make the same mistake. When a paper is longer than the model's fixed input size (its "context window"), the usual fix is to keep the first chunk of text, or to slide over the paper in contiguous chunks. That amounts to assuming the evidence for a relation sits next to where the entities are first mentioned. In full-text papers this assumption fails. The variant is introduced in the methods, the transcription factor that bridges it to the gene appears in the results, and the claim that the variant regulates the gene is stated in the discussion.
We propose to fix this the way chromatin fixes it: fold the document before reading it. For each candidate entity pair, a cheap graph over the whole paper connects mentions, sentences and sections. Short paths through that graph through intermediate "bridge" entities identify a small, non-contiguous set of sentences. These are packed, in their original order, into the fixed window of a strong graph-free extractor. The graph does not do the reasoning. It decides what the transformer gets to see. The claim is that this routing, not more graph layers or a bigger window, accounts for most of the gap between abstract-level and full-text relation extraction. We test it on a new full-text corpus of variant-to-gene mechanism papers and under controlled context budgets on existing benchmarks.
Core Claim
Document-level relation extraction means finding relations whose evidence spans several sentences. The field split into two camps. The first builds explicit document graphs that carry information between distant mentions: Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network, the edge-oriented graphs of Connecting the Dots, and the two-level word/sentence and mention/entity graph of Document-level Relation Extraction with Dual-tier Heterogeneous Graph. The second is represented by ATLOP, which drops the graph and pools the tokens that both entities attend to inside a pretrained encoder. ATLOP beat the graph models on DocRED, CDR and GDA.
Both camps were evaluated almost entirely on inputs that fit in the window. CDR and GDA are PubMed abstracts, and DocRED documents are short Wikipedia passages. So their disagreement says nothing about the regime where the window is the bottleneck. In that regime, SciREX reports that 55% of 4-ary relations in full papers cross section boundaries, and its section-by-section encoder reaches 61.1% F1 even with gold entities.
Our claim is that the two camps solve different subproblems. A heterogeneous graph is the right tool for selecting context, because coreference and bridge-entity paths reach evidence that contiguous chunks miss. Attention pooling is the right tool for reasoning once the evidence is in view. The new element is the interface between them: graph paths decide which sentences enter a fixed-size window, built separately for each entity pair. We predict that this combination degrades much more slowly than truncation or chunking as the window shrinks relative to the document. We also predict the advantage concentrates on pairs whose evidence is linked only through a third entity.
Research Program
The router builds one graph per document. It has mention nodes, entity nodes (mentions merged by string normalization and an off-the-shelf coreference pass), sentence nodes and section nodes. Edges encode same-sentence co-occurrence, mention-to-entity membership, sentence adjacency and section membership. These are the same edge types used by the dual-tier graph, but here they are never passed through a graph neural network.
For a candidate pair, the router enumerates short paths, of at most three entity hops, between any mention of the head and any mention of the tail. It scores each sentence by how many short paths pass through it, with a learned bonus for section type. It then packs the highest-scoring sentences, with their section headers as markers, until the token budget $B$ is full. The extractor is ATLOP, used unchanged.
In a second stage, the sentence scorer is trained from evidence-sentence labels where they exist, which DocRED supplies. The training target is that the packed context contains the gold evidence. The extractor is trained on the packed contexts, so training and test inputs match. Per-pair packing is amortized by building one packed context per head entity and scoring all its tails together, which keeps cost manageable given the pair-count blowup noted for long documents.
The application regime is variant-to-gene regulation in functional genomics papers. This literature motivates the work because its key relations are long-range in two senses at once. Martin et al. report that for over 85% of disease-associated variants, the gene contacted in 3D is not the linearly nearest one. Papers establishing such links, such as RUNX2 Phase Separation Mediates Long-Range Regulation, which links intergenic SNP rs4683184 to XCR1 through RUNX2 condensates, typically state the variant, the bridging transcription factor and the target gene in different sections.
We will annotate about 150 open-access full-text papers of this kind for (variant, gene) regulatory relations, the bridging element or factor, and evidence sentences. The 3D genomics direction then serves as an external consistency check, not as a model input. We will measure what fraction of extracted links name a gene other than the linearly nearest one, and how often extracted pairs agree with tissue-matched chromatin loops in the HUGIn repository. Feeding contact maps into the model would risk a shortcut, because many of these papers derive their claims from exactly such maps.
Baselines
The naive reference point is ATLOP on the first $B$ tokens of the document, the default truncation that most pipelines silently apply. Alongside it we run two "nearest" heuristics that need no learning. The first links each variant to the gene mentioned closest to it in the text, a textual analogue of the nearest-gene rule. The second links it to the genomically nearest gene. Any learned system that fails to beat the textual-nearest heuristic on cross-section pairs has learned little.
The strongest existing reasoning-side competitor is ATLOP itself. It reports 63.4 F1 on DocRED and outperformed the prior best by 4.3 F1 on CDR and 1.4 F1 on GDA (ATLOP). We run it over overlapping sliding windows of size $B$ and max-pool pair logits across windows. This is the fair "chunk everything" version: it uses the same encoder and the same budget per forward pass, with more forward passes, so we report compute alongside accuracy.
The strongest graph-reasoning competitor is the dual-tier heterogeneous graph, which reports 65.9 F1 on CDR, 54.6 inter-sentence F1 on CDR and 83.1 F1 on GDA (Dual-tier Heterogeneous Graph). We give it the same packed or truncated inputs as ATLOP. For full documents we use the SciREX-style section encoder with a sequence model over sections, which reports 61.1% 4-ary F1 with gold entities and 0.8% end-to-end (SciREX). A long-context encoder that reads the whole document is the remaining competitor. No reported result was found on Lacuna for a long-context encoder on SciREX or on our corpus, so we will run one ourselves under the same tuning budget.
The key ablation isolates path routing. It packs sentences that merely contain a mention of the head or the tail, which is plain co-occurrence retrieval, with no bridge-entity paths. An oracle that packs gold evidence sentences gives the upper bound on what routing can achieve. All systems share the encoder, the training data, and a fixed hyperparameter search of equal size.
Beating the competitors means two things. First, graph-routed packing must exceed both sliding-window ATLOP and co-occurrence packing on inter-sentence or cross-section pairs at every budget below the median document length. Second, it must close at least half the gap between truncation and the gold-evidence oracle.
Why not just…?
Why not just use a long-context model that reads the whole paper? This is a real threat for papers that fit in the window. However, Chen et al. show that evidence buried in the middle of long inputs is poorly used even within the window. Context Window Failures in Relational Foundation Models finds that attention over raw, unaggregated relational records collapsed to $R^2 \le 0.18$, while aggregated inputs reached 0.65. We do not treat these results as decisive for text. The long-context run in our experiments settles the question: if it matches routed packing on cross-section pairs at similar compute, the routing contribution is limited to cheaper inference, and we will say so.
Why not just retrieve the passages that mention both entities? This is the most dangerous alternative, because it is a strictly simpler version of our router. It fails by construction when no sentence or passage mentions both entities and the link runs through a bridging factor. That is exactly the variant→TF→gene pattern in mechanism papers, and the multi-hop setting where Groeneveld et al. note that missing one bridging entity sinks the downstream inference. The co-occurrence-packing ablation tests this directly. If it matches path routing even on bridge-only pairs, we conclude that bridge paths are unnecessary and simplify the method accordingly.
Why not just run a deeper graph network over the whole document? Edge-oriented graphs scale quadratically with node count. Graph methods depend on upstream parsers and over-smooth beyond a few layers (Dual-tier Heterogeneous Graph). ATLOP already showed that, when context fits, attention pooling beats graph reasoning. Running the dual-tier model on our packed inputs tells us whether graph reasoning adds anything once routing has supplied the right sentences.
Proposed Experiments
The first experiment is a controlled budget sweep on DocRED, CDR and GDA. We cap the extractor window at $B \in \{64, 128, 256, 512\}$ tokens, which creates truncation pressure on documents whose difficulty is already well characterized. We report overall F1, inter-sentence F1, and evidence recall of the packed context, meaning the fraction of gold evidence sentences that made it into the window. Evidence recall separates selection failures from reasoning failures. The prediction is that truncation and sliding-window F1 fall steeply as $B$ shrinks while routed packing stays close to the oracle. The decisive pattern is a widening gap on inter-sentence pairs.
The second experiment uses real full texts. On SciREX we use the gold-entity setting so that entity detection does not contaminate the comparison, and compare against the reported 61.1% F1, with relations stratified by whether they cross sections. On the new variant-to-gene corpus we evaluate end-to-end with an off-the-shelf biomedical entity tagger. We stratify by whether the bridging entity is the only connection between variant and gene, and report the share of correct links that are not the textual-nearest or genomic-nearest gene, along with agreement with HUGIn loops.
The idea fails if co-occurrence packing matches path routing on bridge-only pairs, or if sliding-window ATLOP matches it at equal compute. The idea holds if routed packing wins specifically on cross-section, bridge-mediated relations, with evidence recall explaining the win.
References
- Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network
- Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs
- Document-level Relation Extraction with Dual-tier Heterogeneous Graph
- Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling
- SciREX: A Challenge Dataset for Document-Level Information Extraction
- RUNX2 Phase Separation Mediates Long‐Range Regulation Between Osteoporosis‐Susceptibility Variant and XCR1 to Promote Osteoblast Differentiation
- GuARD: Effective Anomaly Detection through a Text-Rich and Graph-Informed Language Model
- Context Window Failures in Relational Foundation Models
- A Simple Yet Strong Pipeline for HotpotQA