Noise Reduction Methods for Distantly Supervised Biomedical Relation Extraction

Gang Li, Cathy H. Wu, K. Vijay-ShankerBioNLP 2017 · OpenReview

Abstract: Gang Li, Cathy Wu, K. Vijay-Shanker. BioNLP 2017.

2017.

Introduction

In the biomedical domain, the rapid growth of scientific literature necessitates automated methods for extracting relationships between entities, such as protein-protein interactions or gene regulations. Distant supervision (DS) has emerged as a powerful technique to generate large-scale training data by aligning existing knowledge bases with unlabeled text. However, DS relies on the strong assumption that any sentence containing a known entity pair expresses their relationship, leading to significant "noise" in the form of false positives and false negatives. This paper presents three novel heuristics designed to mitigate this noise and improve the performance of biomedical relation extraction models.

Methodology

The authors propose a pipeline of three heuristics—Closest Pairs (CP), Trigger Words (TW), and High-Confidence Patterns (HP)—to refine both the positive ($P$) and negative ($N$) training sets.

1. Heuristic of Closest Pairs (CP)

When a sentence contains multiple mentions of the same entities, the CP heuristic assumes that only the pair with the shortest dependency path is likely to express the relation. By focusing on the most grammatically direct connection, the model filters out secondary mentions that often introduce noise.

Example of dependency paths between entities used to identify the closest pair and syntactic relations.

2. Heuristic of Trigger Word (TW)

Syntactic structures alone are sometimes insufficient. The authors automatically mine "trigger words" (e.g., bind, regulate) that are statistically significant indicators of a relationship. If a candidate sentence in the positive set does not contain a trigger word along its dependency path, it is discarded as a false positive.

3. Heuristic of High-Confidence Patterns (HP)

While the first two heuristics clean the positive set, the HP heuristic addresses the negative set. It identifies frequent grammatical patterns from the cleaned positive set and uses them to scan the negative set. If a "negative" sentence matches a high-confidence positive pattern, it is removed, as it likely represents a true relationship missing from the knowledge base (a false negative).

Experimental Setup and Datasets

The proposed methods were evaluated across three distinct biomedical tasks:

  • PPI (Protein-Protein Interaction): Utilizing the IntAct database and AIMed corpus.
  • MIRGENE (miRNA-gene regulation): Utilizing Tarbase/miRTarBase and a Medline-sampled corpus.
  • PLOC (Protein-localization): Utilizing UniProt and the BioNLP 2011 Genia dataset.

Results and Discussion

The application of these heuristics led to substantial performance improvements over baseline Logistic Regression and Multi-Instance Learning models. The F-score increased by 6 points for PPI, 10 points for MIRGENE, and 14 points for PLOC.

One of the most significant findings is the impact of cleaning the negative set. As shown in the performance plots, using even a modest number of high-confidence patterns (50–100) to filter the negative set consistently improves recall and overall F-score across all three datasets.

Performance metrics (F-score, Precision, Recall) across PPI, MIRGENE, and PLOC datasets relative to the number of high-confidence patterns used to clean the negative set.

Furthermore, the models achieved high precision at lower recall levels (e.g., 0.95 precision for MIRGENE at 0.30 recall), making them highly suitable for assisting human curators in database population tasks.

Conclusion

This research demonstrates that addressing noise in both positive and negative training sets is crucial for effective distant supervision. By leveraging syntactic dependency paths and statistical trigger words, the proposed heuristics significantly enhance the reliability of extracted biomedical relations. Future work will explore aggregating evidence across multiple sentences and incorporating human-in-the-loop refinement for trigger word lists.

Summary updated

BibTeX
@inproceedings{li2017noise,
  author = {Gang Li and Cathy H. Wu and K. Vijay-Shanker},
  title = {{Noise Reduction Methods for Distantly Supervised Biomedical Relation Extraction}},
  booktitle = {BioNLP},
  year = {2017},
  url = {https://openreview.net/forum?id=YVTuIDUTt5R},
}
Figure