Parser-Free Relation Extraction Trained on Document Dependency Paths

Finding relations between entities in a long document, such as "chemical X causes disease Y" in a biomedical abstract, requires the model to link evidence that is spread across sentences. Two families of methods compete here. One builds explicit structure: syntax trees, entity–sentence graphs, and the shortest dependency path, which is the chain of grammatically linked words connecting two entities in a parse tree. The other argues that a pretrained transformer's own attention already does this work. The structural family has the better numbers on DocRED, a large document-level benchmark. It also depends on an external parser at test time, so parser mistakes feed directly into predictions. The attention family is simpler and nearly as accurate, but it gets no guidance about which words actually carry a relation.

We propose to use the structure only as a teacher. For each entity pair we compute a document dependency path: a shortest path through a heterogeneous document graph, meaning a graph with several kinds of nodes and edges, here intra-sentence dependency edges, coreference links, and links between entity mentions. The path's tokens become a soft target for the transformer's entity-pair attention during training. Because a single parse may be wrong, the target is spread over several parses, and it is refined in alternating rounds in which the model's own attention re-scores competing paths. At test time only the transformer and a pooling head run, with no parser and no graph. If this works, it settles an open dispute. It would show that explicit syntax and document graphs help mainly by shaping what the language model attends to, and that this help can be kept after the parser is removed.

Core Claim

A graph-free document relation extractor, trained so that its entity-pair attention concentrates on document-level shortest-dependency-path tokens, will beat the same model without this signal. Its gain will be concentrated on inter-sentence relations and biomedical text. It will match or exceed parser-dependent graph models while running no parser at inference. The new ingredient is not the shortest dependency path itself; methods going back to Bunescu and Mooney use it. Nor is it the document graph, since EoG and DHG build such graphs. The new ingredient has three parts. First, the path is extended across sentence boundaries through the document graph. Second, it is used as an uncertain, revisable supervision target rather than as a hard input filter. Third, the language model's attention feeds back into which path counts as correct.

Relation to Prior Work

The motivating tension is between graph-free and parser-dependent methods. ATLOP is graph-free. It pools each entity pair's context from the transformer's own attention weights ("localized context pooling") and uses a learned per-pair threshold. It reports 63.4 F1 on DocRED and argues that graph networks may be overkill. FCDS takes the opposite route. It fuses constituency trees with dependency graphs joined by a document node, runs on DeBERTa-Large, and reports 64.03% IgnF1 on DocRED. It also improves on CDR and GDA, two biomedical benchmarks, but it needs parsers at test time. We take ATLOP's inference architecture and FCDS's structural knowledge. We ask whether the second can be moved entirely into training.

Two works constrain the design. LSR is the closest prior work. It already places shortest-dependency-path tokens as "meta dependency path" nodes in a latent document graph. However, its BERT encoder is decoupled from graph refinement, and its path nodes come from a fixed external parse. It reports 59.05 F1 on DocRED with BERT-base. DHG's authors explicitly name CoreNLP parse errors as degrading their structural layer. The parser-brittleness analysis in Direction 27072 makes the same point more generally. These results rule out designs that trust a single parse at test time.

Four works supply method ingredients. RbSP showed at the sentence level that pure paths drop negations and modifiers. Restoring attended child nodes was worth 1.22 F1 on SemEval-2010 Task 8, so our target also gives weight to path children. GLEM alternates between training a language model and a graph model, each supplying pseudo-labels for the other. It found that the language model internalized structure well enough to predict without edges at inference. That result is the direct precedent for our "structure at training only" bet, although GLEM worked on citation-graph node classification rather than on syntax inside documents. DEPTH shows that path-based sentence simplification sharply reduces hallucination in large-language-model relation extraction. It does so as an immutable, single-sentence preprocessing step, which is exactly what we avoid.

Research Program

Building the teacher graph. For each training document, we build a graph whose nodes are tokens and whose edges are of three kinds:

  • dependency arcs within each sentence;
  • coreference and same-entity links between mentions;
  • a weak edge between the syntactic roots of adjacent sentences.

The document dependency path for entities $s$ and $o$ is a lowest-cost path from any mention of $s$ to any mention of $o$. Edge costs depend on edge type, so that crossing sentences is discouraged unless coreference provides a bridge. For intra-sentence pairs this reduces to the classic shortest dependency path. For inter-sentence pairs it passes through bridge entities, which is the multi-hop routing that EoG and DHG encode with sentence hub nodes.

Handling parser uncertainty. We parse each sentence with several parsers or with $K$-best parses. Each token $t$ gets a prior weight $q_0(t)$. This weight is the fraction of parse–path combinations containing $t$. Children of path tokens receive a reduced weight $\gamma$, following RbSP.

Training objective. Let $a_{so}(t)$ be ATLOP's localized context attention for pair $(s,o)$ over token $t$. This is the product of the two entities' attention rows, renormalized. The training loss is

$$ \mathcal{L} = \mathcal{L}_{\text{AT}} + \lambda \sum_{(s,o)} \mathrm{KL}\big(q_{so} \,\|\, a_{so}\big), $$

where $\mathcal{L}_{\text{AT}}$ is ATLOP's adaptive-threshold classification loss, $q_{so}$ is the normalized path-token target, and $\lambda$ sets the strength of the structural signal. The penalty applies only to pairs with a gold relation. For "no relation" pairs there is no evidence to point to.

Refining the target. After each training round we re-score each candidate path $p$. The new score is its parser prior multiplied by $\exp(\beta\, m(p))$, where $m(p)$ is the attention mass the current model places on $p$'s tokens and $\beta$ controls how much the model's view counts. The target $q_{so}$ is then rebuilt from the re-scored paths. This is a GLEM-style alternation. The parser proposes paths, and the language model can down-weight a path that the rest of the document contradicts, which is where parser error correction would come from. We cap refinement at three rounds, following LSR's observation that refinement peaks at two to three iterations. At inference, $q$ is never computed.

Baselines

Naive reference point: ATLOP. We re-run ATLOP with our backbone, seeds, and tuning budget. The proposal is literally ATLOP plus one loss term, so any gain is attributable to the structural target. The published reference is 63.4 F1 on DocRED (ATLOP). For CDR and GDA the file reports only relative gains (+4.3 and +1.4 F1 over prior models), and no absolute ATLOP number was found on Lacuna. We therefore report our reproduced numbers.

Strongest existing method: FCDS. FCDS reports 64.03% IgnF1 on DocRED with DeBERTa-Large. IgnF1 is F1 that ignores facts already seen in the training set. We compare against it with the same DeBERTa-Large backbone and the same hyperparameter search budget. FCDS runs its parsers at test time; ours does not. Beating it means exceeding its IgnF1 averaged over five seeds, with a confidence interval that excludes it. Merely matching it without a parser would still be a meaningful result, because it shows the parser can be removed without cost.

Biomedical graph baselines: DHG and EoG. On CDR the file reports overall and inter-sentence numbers. DHG reaches 65.9% F1 overall and 54.6% on inter-sentence relations, and 83.1% on GDA. EoG reaches 63.6% overall and 50.9% inter-sentence on CDR. We cite these numbers and, where code allows, re-run them on our backbone.

Latent-graph baseline: LSR. LSR reports 59.05 F1 on DocRED with BERT-base. It is compared at the BERT-base scale. It is the closest conceptual competitor because it also uses path tokens inside a document graph.

Ablations. Four ablations isolate the key ingredient:

  • Sentence paths only. The target uses sentence-level paths with no coreference bridges. This tests whether the document graph matters.
  • Random tokens. The target is a random token set of matched size. This rules out a generic attention-regularization effect.
  • Single parse, no refinement. A single one-best parse with no refinement rounds. This tests whether handling parser uncertainty matters.
  • Parser at inference. The same path is used as a hard attention mask at test time. This measures what, if anything, is lost by dropping the parser.

Why not just…?

Why not just use ATLOP, since attention already learns syntax? This is the most serious threat, because ATLOP beat earlier graph models. Our answer is empirical. The comparison is exactly ATLOP versus ATLOP plus the path loss, on identical settings. If the gain is under one seed standard deviation on DocRED and also absent on the inter-sentence split of CDR, we will conclude that ATLOP's simplicity argument holds, and report it. FCDS's margin over ATLOP-class models suggests there is signal left for explicit structure to add. That margin is small, though, which is why we separately report the inter-sentence and biomedical splits, where parser-derived structure should matter most.

Why not just feed parser-derived structure into a graph network on frozen features, as DHG and LSR do? These cascaded designs keep the parser at test time and inherit its errors, which DHG's own authors acknowledge. They also cannot let the language model revise the structure. We will test this directly with controlled parse corruption. We perturb a fixed fraction of dependency arcs at test time for parser-dependent models, and only at training time for ours, and plot F1 against the corruption rate. Our model should degrade less steeply.

Why not just prune the input text to the path, as DEPTH and depLCNN do? Hard pruning helps at the sentence level (depLCNN, 85.6 F1 on SemEval-2010 Task 8). However, a document has many entity pairs, so each pair would need its own pruned input. Pruning also discards context RbSP showed to be necessary. We include a pruned-input variant, in which each pair's input is its document path plus children, to measure this cost directly.

Why not just add the attention loss on sentence-level paths and skip the document graph? This is plausible, and it is one of our ablations. If sentence-only targets match the full method on inter-sentence relations, the document-graph component is unnecessary and the contribution shrinks to "syntax as attention supervision." We would report that honestly.

Proposed Experiments

Datasets and metrics. We train and evaluate on DocRED (DocRED), CDR, and GDA. We report F1 and IgnF1, split into intra- and inter-sentence pairs, averaged over five seeds, at BERT-base, RoBERTa-large, and DeBERTa-Large scales so that each published baseline has a matched comparison. On SemEval-2010 Task 8 we run a sanity check: our loss applied to a sentence-level transformer should approach RbSP's 86.3 F1 without using paths at inference.

Decisive checks. Three results decide the outcome:

  • Gains concentrated on inter-sentence pairs, and absent under the random-token control, would show that the document path carries information attention does not find alone.
  • A flatter degradation curve under parse corruption would show that removing the parser at inference, together with the refinement rounds, delivers robustness.
  • An increasing overlap between the model's attention and paths from a held-out, higher-accuracy parser across refinement rounds would show that the language model is correcting parses rather than simply copying them.

If all three hold, explicit document syntax becomes something paid for once at training time. If the first fails, the evidence favors ATLOP's position, and that is also worth publishing cleanly.