Abstract
Semi-supervised bootstrapping techniques for relationship extraction from text iteratively expand a set of initial seed instances. Due to the lack of labeled data, a key challenge in bootstrapping is semantic drift: if a false positive instance is added during an iteration, then all following iterations are contaminated. We introduce BREX, a new bootstrapping method that protects against such contamination by highly effective confidence assessment. This is achieved by using entity and template seeds jointly (as opposed to just one as in previous work), by expanding entities and templates in parallel and in a mutually constraining fashion in each iteration and by introducing higher-quality similarity measures for templates. Experimental results show that BREX achieves an F1 that is 0.13 (0.87 vs. 0.74) better than the state of the art for four relationships.
Introduction
Relation Extraction (RE) is the task of identifying structured relationships between entities in unstructured text—for instance, recognizing that "Google acquired YouTube" implies an acquired relationship. While supervised deep learning models are highly effective, they require massive amounts of hand-labeled data, which is often prohibitively expensive to produce.
To address this, researchers utilize bootstrapping: starting with a few "seeds" (e.g., the pair <Google, YouTube>), the system finds where they appear in a corpus, extracts the linguistic patterns around them (templates), and uses those templates to discover new entity pairs. The primary challenge of this approach is semantic drift. If a system mistakenly identifies a noisy pattern (e.g., "X met with Y") as a valid template for an acquisition, it will begin extracting unrelated pairs. Once a false positive enters the seed pool, the system quickly "drifts" away from the target relation, corrupting all subsequent learning.
The BREX Framework
This paper introduces BREX (Bootstrapping RE eXtraction), which mitigates semantic drift through three major innovations:
1. Joint Learning (BREJ)
Unlike prior systems that bootstrap using only entity pairs (BREE) or only templates (BRET), BREJ uses both. It treats them as mutually constraining: an extraction is only considered high-confidence if it aligns with known good entities or known good templates. This "disjunctive matching" creates a higher barrier for noisy data to enter the seed set.
2. The Three-Hop Strategy
To balance recall and precision, the algorithm performs three distinct "hops" from the seeds in each iteration:
- Hop-1: Find instances in the corpus similar to the current seeds.
- Hop-2: Cluster these instances into "extractors."
- Hop-3: Identify all instances in the corpus that match these new extractors.
3. Cross-Context Attentive Similarity ($sim_{cc}$)
Traditional systems often focus on the text between two entities. However, valuable relational information often resides in the prefixes or suffixes of the entity mentions. The authors' new similarity measure looks across all parts of the context (before, between, and after the entities) to find matches, even when syntax varies significantly.
Experimental Setup
The researchers evaluated BREX using a corpus of 5.5 million news articles from the English Gigaword (AFP and APW). They focused on a subset of 1.2 million sentences containing at least two entities linked to FreebaseEasy, targeting four specific relationships: acquired (ORG-ORG), founder-of (ORG-PER), headquartered (ORG-LOC), and affiliation (ORG-PER).
Key Results
The experimental results demonstrate that the joint approach (BREJ) significantly outperforms isolated bootstrapping methods. While BRET (templates only) maintains high precision, it suffers from low recall. BREJ manages to retain the high precision of templates while using entity pairs to drive significantly higher recall.
As shown in the results, BREJ achieved an average F1 score of 0.87, compared to the previous state-of-the-art (BREE/BREDS) at 0.74. This 13% absolute improvement highlights the effectiveness of using templates as anchors to license new entity pairs. Furthermore, the $sim_{cc}$ measure allowed the authors to eliminate four hyper-parameters related to context weighting, making the system easier to tune than previous models.
Conclusion
BREX provides a robust framework for relation extraction that effectively counters the problem of semantic drift. By leveraging joint learning and a holistic view of sentence context, it achieves state-of-the-art performance with minimal supervision. Future research may explore the system's dependency on initial seed quality and its applicability to more informal or technical domains.
Summary updated