Abstract: Gang Li, Cathy Wu, K. Vijay-Shanker. BioNLP 2017.
2017.
Introduction
In the biomedical domain, the rapid growth of scientific literature necessitates automated methods for extracting relationships between entities, such as protein-protein interactions or gene regulations. Distant supervision (DS) has emerged as a powerful technique to generate large-scale training data by aligning existing knowledge bases with unlabeled text. However, DS relies on the strong assumption that any sentence containing a known entity pair expresses their relationship, leading to significant "noise" in the form of false positives and false negatives. This paper presents three novel heuristics designed to mitigate this noise and improve the performance of biomedical relation extraction models.
Methodology
The authors propose a pipeline of three heuristics—Closest Pairs (CP), Trigger Words (TW), and High-Confidence Patterns (HP)—to refine both the positive ($P$) and negative ($N$) training sets.
1. Heuristic of Closest Pairs (CP)
When a sentence contains multiple mentions of the same entities, the CP heuristic assumes that only the pair with the shortest dependency path is likely to express the relation. By focusing on the most grammatically direct connection, the model filters out secondary mentions that often introduce noise.
2. Heuristic of Trigger Word (TW)
Syntactic structures alone are sometimes insufficient. The authors automatically mine "trigger words" (e.g., bind, regulate) that are statistically significant indicators of a relationship. If a candidate sentence in the positive set does not contain a trigger word along its dependency path, it is discarded as a false positive.
3. Heuristic of High-Confidence Patterns (HP)
While the first two heuristics clean the positive set, the HP heuristic addresses the negative set. It identifies frequent grammatical patterns from the cleaned positive set and uses them to scan the negative set. If a "negative" sentence matches a high-confidence positive pattern, it is removed, as it likely represents a true relationship missing from the knowledge base (a false negative).
Experimental Setup and Datasets
The proposed methods were evaluated across three distinct biomedical tasks:
- PPI (Protein-Protein Interaction): Utilizing the IntAct database and AIMed corpus.
- MIRGENE (miRNA-gene regulation): Utilizing Tarbase/miRTarBase and a Medline-sampled corpus.
- PLOC (Protein-localization): Utilizing UniProt and the BioNLP 2011 Genia dataset.
Results and Discussion
The application of these heuristics led to substantial performance improvements over baseline Logistic Regression and Multi-Instance Learning models. The F-score increased by 6 points for PPI, 10 points for MIRGENE, and 14 points for PLOC.
One of the most significant findings is the impact of cleaning the negative set. As shown in the performance plots, using even a modest number of high-confidence patterns (50–100) to filter the negative set consistently improves recall and overall F-score across all three datasets.
Furthermore, the models achieved high precision at lower recall levels (e.g., 0.95 precision for MIRGENE at 0.30 recall), making them highly suitable for assisting human curators in database population tasks.
Conclusion
This research demonstrates that addressing noise in both positive and negative training sets is crucial for effective distant supervision. By leveraging syntactic dependency paths and statistical trigger words, the proposed heuristics significantly enhance the reliability of extracted biomedical relations. Future work will explore aggregating evidence across multiple sentences and incorporating human-in-the-loop refinement for trigger word lists.
Summary updated