Target Aware Attention for Noisy Relation Extraction

This direction is split into other directions: multi-instance attention. Its last version stays readable and citable here.

Version 1 of 1134 papers154 conceptsfrozen 2026-10-04

In the effort to build massive knowledge bases, researchers often rely on distant supervision: the assumption that if two entities (like "Steve Jobs" and "Apple") have a known relationship in a database, any sentence containing both entities likely expresses that relationship. While this allows for the creation of huge datasets without manual labeling, it introduces a significant amount of "noise." Many sentences containing the entities are irrelevant or describe different facts entirely.

To solve this, a research direction has emerged centered on target-aware attention. Instead of treating every word or sentence with equal importance, these models use the entities or relations themselves as "queries" to selectively filter the signal from the noise.

The Problem of the Noisy Bag

In distant supervision, training data is organized into "bags." A bag contains all sentences from a corpus that mention a specific entity pair. The standard assumption—pioneered in earlier work—was that at least one sentence in the bag expresses the target relation. However, traditional models often struggled because they either picked only the single "best" sentence or averaged all sentences together, allowing noisy, irrelevant sentences to dilute the learning signal.

Yankai Lin, Zhiyuan Liu, and their colleagues addressed this in Neural Relation Extraction with Selective Attention over Instances. Their insight was to use an instance-level attention mechanism. By calculating a weight for each sentence based on how well it matches the representation of the target relation, the model can dynamically "de-emphasize" noisy sentences. If a sentence in the "Steve Jobs/Apple" bag is actually about a product launch, it gets a high weight; if it’s just a list of famous people that happens to include Jobs, it gets a low weight.

Attention as a Semantic Query

The intuition behind this approach is that the target (the entity or the relation) acts as a search query. Just as a search engine filters the web for relevance, target-aware attention filters a sentence or a bag for semantic alignment.

This concept was extended to the word level to handle "inner-sentence noise." Even within a relevant sentence, many words are filler. Xiaocheng Feng and Bing Qin introduced Effective Deep Memory Networks for Distant Supervised Relation Extraction, which uses a dual-attention strategy. It first identifies important words within a sentence (word-level attention) and then identifies important sentences within a bag (instance-level attention).

Similarly, Guoliang Ji and Jun Zhao demonstrated in Distant Supervision for Relation Extraction with Sentence-Level Attention and Entity Descriptions that incorporating external information, such as entity descriptions from Wikipedia, can further sharpen this attention. By knowing what an entity is, the model becomes better at judging which context words are relevant to its relationships.

Beyond Simple Pairs: Context and Structure

Sentences rarely contain just one fact. Daniil Sorokin and Iryna Gurevych argued in Context-Aware Representations for Knowledge Base Relation Extraction that the "context relations"—other facts mentioned in the same sentence—provide vital clues. They used attention to selectively incorporate these auxiliary relations, recognizing that the presence of a "founded by" relation in a sentence makes the presence of an "employer" relation more likely.

For complex cases where a single sentence contains multiple overlapping relations, Yue Yuan and Shirui Pan proposed the Relation-Specific Attention Network (RSAN). Instead of one general representation for a sentence, RSAN generates a specialized representation for every possible relation. It asks: "If I am looking for the 'Founder' relation, which words in this sentence matter?" and then asks: "If I am looking for the 'CEO' relation, which words matter now?"

Generalizing the Target Aware Framework

While born in Relation Extraction, this "query-based filtering" has proven universal across Information Extraction tasks:

Why It Works: The Intuition of Focus

The success of target-aware attention lies in its move away from static representations. In early deep learning models, a sentence had one vector. In target-aware models, a sentence has a distribution of potential meanings that are unlocked by the target query.

By allowing the entities and relations to dictate which parts of the text are informative, these models effectively simulate a human reader's ability to skim a noisy document for specific facts. As Tianyi Liu and Weijia Jia summarized in their work on Inner-Sentence Noise Reduction, the goal is to ensure the subject-object pair remains the primary signal, allowing the model to remain robust even when the surrounding text is cluttered or misleading.

Papers in this version

Showing 60 of 134 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_target_aware_attention_for_noisy_relatio_95a24e9b,
  title = {Target Aware Attention for Noisy Relation Extraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/target-aware-attention-for-noisy-relation-extraction/rdr_95a24e9b646e484bb1757eeb3c2009fd},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_target_aware_attention_for_noisy_relatio_95a24e9b_v1,
  title = {Target Aware Attention for Noisy Relation Extraction},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/target-aware-attention-for-noisy-relation-extraction/rdr_95a24e9b646e484bb1757eeb3c2009fd/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}