When training machine learning models to extract relationships from text, natural language processing (NLP) researchers face a fundamental challenge: noise. A single sentence can be packed with clauses, appositives, and descriptive adjectives that are entirely irrelevant to the core fact you want to extract.
To combat this, early relation extraction and information retrieval models relied on aggressive pruning heuristics. The most elegant of these was the Shortest Dependency Path (SDP). By converting a sentence into a grammatical dependency tree and tracing the minimal path of edges connecting two target entities, models could discard the rest of the sentence as irrelevant noise. For instance, the classic SDP-LSTM framework by Xu et al. demonstrated that focusing strictly on this core path could dramatically streamline relation classification. A parallel strategy dominated Information Retrieval (IR), where common "stop words" and function words (like to, from, where, and not) were systematically purged to optimize index efficiency and focus on heavy, content-bearing nouns.
However, this elegant simplification harbors a severe structural blindspot. By treating everything outside the minimal skeleton as noise, these models inadvertently throw away the exact linguistic signals required to accurately interpret complex text. Recent research has shifted toward diagnosing this contextual information loss and designing architectures that preserve these vital peripheral signals.
The Hidden Value of Function Words
The intuition behind removing function words or stop words is that they carry little semantic weight on their own. While a noun like "penicillin" or "patient" has a clear real-world referent, a preposition like "by" or "with" feels empty.
But in structural syntax, these words function as critical logical connectors and direction markers. When they are stripped away, the underlying logic of an assertion frequently collapses:
- Directionality and Inverse Relations: Consider trying to determine the direction of a relationship between two entities. As noted by Wu et al., function words and phrase-level cues are the primary signals that establish directionality. Without them, a model cannot reliably distinguish a relation from its inverse (e.g., distinguishing whether Entity A managed Entity B, or vice versa), particularly in low-resource settings where data is scarce.
- Domain-Specific Nuance: In specialized literature, such as medical text, function words are far from noise. Patel et al. demonstrated that multi-word syntactic patterns built around function words (such as patients with versus treated with) provide the essential structural signaling required to differentiate a patient's underlying condition from their intervention.
- Intent Resolution: In information retrieval, conventional search engines frequently prune words like where or to. However, Dai et al. showed that modern contextual language models achieve deeper text understanding precisely because they retain these words, leveraging them to resolve complex natural language intent that keyword matching misses.
The Structural Blindspot of the Shortest Path
If filtering individual words causes information loss, pruning entire sentences down to the SDP compounds the problem. While the SDP successfully captures long-range interactions between distant entities, it suffers from two fatal flaws: it creates semantic ambiguity, and it systematically excludes critical modifiers.
1. The Problem of Identical Paths
The shortest path between two words is a minimalist chain of grammatical dependencies. The issue is that two sentences can feature identical grammatical chains while expressing entirely different meanings. As argued by Ji et al., a bare verb-object path lacks the context needed to distinguish between semantically distinct relationships. The true meaning of that path often hinges entirely on the subtrees and modifiers attached to the words within the path—context that the SDP completely discards.
2. Catastrophic Omissions (Negation and Modality)
Because the SDP only cares about the absolute shortest bridge between Entity A and Entity B, any word that branches off this direct line is ignored. This means that critical logical qualifiers are routinely dropped.
As highlighted by Zhang et al., rigid tree-pruning strategies systematically exclude modifiers like negations (not) or conjunctions (and). If a sentence reads, "The drug did not cause the side effect," the SDP between drug and side effect will typically pass through cause, completely missing the word not branching off to the side. The model is left with an anemic representation that extracts a positive causal relationship—a catastrophic factual inversion.
[cause] <-- Shortest Dependency Path stops here
/ \
[drug] [side effect]
|
[not] <-- Crucial modifier is discarded as "noise"
Similarly, Can et al. pointed out that adverbs and peripheral qualifiers are routinely left behind by the SDP, stripping away the essential semantic context required to correctly identify how, when, or under what conditions a relationship occurs.
Evolving Beyond Hard Pruning
To resolve this tension between noise reduction and information loss, modern architectures have evolved away from rigid, rule-based "hard pruning" toward flexible, context-aware structures. The goal is to build models that can dynamically determine which peripheral words matter.
Augmented Paths and Syntactic Neighborhoods
Instead of restricting inputs to a bare skeletal path, researchers began designing frameworks that explicitly pull the immediate context back in. Ji et al. introduced the Augmented Dependency Path (ADP), which attaches the local subtrees of path nodes to the main sequence.
To make this process more selective, Can et al. developed a "Richer-but-Smarter" SDP (RbSP) framework. Rather than blindly appending every nearby node, it uses a multi-layer attention mechanism to scan the surrounding dependency tree and selectively harvest only the child nodes that contain relevant modifiers.
Furthermore, models must understand the grammatical environment surrounding the entities themselves. Zhang et al. proposed Syntax-Aware Entity Embeddings (SEE), which utilize tree-structured neural networks to encode the "syntactic neighborhoods" rooted at each entity. This ensures that local modifiers and qualifiers are baked directly into the entity representations before relation classification even begins.
Soft Pruning via Attention-Guided Graphs
The ultimate realization of this shift is the complete abandonment of static, human-defined pruning rules in favor of learnable, continuous structures.
Hard Pruning (SDP): [Node] ---- (Keep) ---- [Node] [Node] ---- (Drop) ---- [Modifier]
Soft Pruning (GCN): [Node] ---- (W = 0.9) -- [Node] [Node] ---- (W = 0.4) -- [Modifier]
Instead of manually cutting edges to simplify a dependency tree, Guo et al. introduced Attention Guided Graph Convolutional Networks (AGGCNs). This architecture maintains the full sentence graph but uses multi-head self-attention to assign a continuous weight to every single edge.
By training the model to adjust these weights end-to-end, the network effectively performs "soft pruning." It retains the capacity to pass information across long-range paths, but it can dynamically dial up the attention on a distant negation modifier or dial down the weight on an irrelevant appositive clause. This architecture bridges the gap: it maintains the computational focus of a pruned path while ensuring that no critical contextual signal is ever permanently locked out.
Key Foundational Work
Filtrages syntaxiques de co-occurrences pour la représentation vectorielle de documents (2002) — contributed 2 concept elements to this direction.
Subdeletion in Verb Phrase Ellipsis (1996) — contributed 1 concept elements to this direction.