When humans read a sentence like "The legendary studio, which had faced bankruptcy twice before being acquired by an international conglomerate, finally released its highly anticipated sequel," we instantly grasp the core event: acquired_by(studio, conglomerate). However, for a machine learning model, the sentence is packed with distracting details—adverbs, relative clauses, and parenthetical descriptions.
This extraneous text is known as contextual noise. As ChengXiang Zhai et al. demonstrated in A Systematic Exploration of the Feature Space for Relation Extraction, non-essential tokens distant from target entities introduce significant noise that obscures meaningful semantic patterns. When models process entire sentences indiscriminately, they risk overfitting to these irrelevant background structures, mistaking coincidental co-occurrences for genuine relational signals.
To combat this, a major research direction focuses on isolating the "semantic skeleton" of a sentence while discarding the linguistic fluff.
Finding the Structural Backbone via the Shortest Dependency Path
To prevent models from drowning in contextual noise, researchers turned to syntax. A dependency tree maps grammatical relationships between words, bypassing their linear order. By isolating the Shortest Dependency Path (SDP) between two target entities, models can effectively filter out peripheral modifiers.
[studio] <--nsubjpass-- [acquired] --agent--> [conglomerate]
|
(which had faced...) (by an international...)
By focusing only on the words directly on this grammatical bridge (studio $\leftarrow$ acquired $\rightarrow$ conglomerate), the model completely ignores the noisy clause about bankruptcy.
This intuition formed the basis of several foundational neural architectures. Kun Xu 0005 et al. applied Convolutional Neural Networks to the SDP in Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling, demonstrating that stripping away complex clauses drastically improves relation classification. Simultaneously, Yan Xu et al. utilized LSTMs along the SDP in Classifying Relations via Long Short Term Memory Networks along Shortest Dependency Paths to capture directional semantic links without distraction from irrelevant words. Restricting the model's view to the minimal grammatical path shrinks the hypothesis space, forcing the network to focus exclusively on the relational triggers.
Beyond Rigid Paths: Dynamic Pruning and Structural Blocks
While the SDP is a powerful filter, rigid adherence to it can sometimes discard vital context, such as negation or modal verbs that modify the relation. To address this, more flexible structural filtering methods emerged.
Instead of relying on a strict one-word-wide path, Dongsheng Wang et al. proposed isolating "structural blocks"—the essential tokens immediately surrounding entities—and ignoring the rest of the sentence in Structural block driven enhanced convolutional neural representation for relation extraction.
Rather than relying on static rules, other approaches let the data dictate what to keep. In Learning to Prune Dependency Trees with Rethinking for Neural Relation Extraction, Bowen Yu et al. developed a method to dynamically learn how to prune dependency trees. Their approach allows the model to adaptively determine which grammatical connections act as noise and which are essential for resolving the core semantic relationship, combining the benefits of structural guidance with data-driven flexibility.
Latent Denoising: Attention and Rationale Extraction
What if we want to avoid external syntactic parsers, which can be error-prone or unavailable for low-resource languages? Another branch of research focuses on latent denoising directly within neural architectures, using attention and sparsity constraints to ignore distractors.
In Multi-level Attention-Based Neural Networks for Distant Supervised Relation Extraction, Linyi Yang et al. introduced multi-level attention mechanisms operating at both the word and sentence levels to automatically down-weight non-informative filler words. Taking this a step further, Xuming Hu et al. proposed the $RE^2$ framework in Think Rationally about What You See: Continuous Rationale Extraction for Relation Extraction. This framework enforces sparsity and continuity constraints to extract explicit "rationales" from text. By training the model to select only the most informative continuous segments, it naturally filters out inner-sentence distractors, preventing the model from latching onto spurious correlations.
The Semantic Skeleton in Large Language Models
The challenge of contextual noise does not disappear with scale. Even as the field has transitioned toward in-context learning with Large Language Models (LLMs), long and verbose contexts continue to introduce interference that degrades performance.
To mitigate this in modern architectures, Peitao Han et al. introduced AMR-RE: Abstract Meaning Representations for Retrieval-Based In-Context Learning in Relation Extraction. This framework uses Abstract Meaning Representation (AMR) graphs to isolate the "semantic skeleton" of sentences. By stripping away surface-level linguistic fluff before retrieving few-shot examples for an LLM, they demonstrate that explicit noise reduction remains a critical mechanism for guiding even the most advanced language models toward accurate relational inference.
Key Foundational Work
Using Syntactic Dependency as Local Context to Resolve Word Sense Ambiguity (1997) — contributed 1 concept elements to this direction.