Distant Supervision for Cancer Pathway Extraction from Text

Hoifung Poon, Kristina Toutanova, Chris Quirk

Abstract

Mapping the biological pathways of cancer—the complex "wiring diagrams" of protein and gene interactions—is critical for oncology research. However, much of this knowledge is locked in millions of unstructured scientific abstracts. This paper presents an automated approach using "distant supervision" to extract pathway interactions from 22 million PubMed abstracts. By leveraging existing databases to automatically label training data, the researchers bypassed the manual annotation bottleneck, extracting 1.5 million interactions with performance approaching that of traditional supervised learning.

Introduction

Understanding cancer requires a comprehensive map of biological pathways: the chain reactions controlling cell behavior. While thousands of new papers are published daily, the knowledge remains trapped in text. Traditional extraction methods—manual curation, rule-based systems, or supervised machine learning—face significant hurdles. Manual curation is too slow, rule-based systems are too rigid, and supervised learning requires expensive, expert-labeled datasets.

The primary challenge lies in the linguistic diversity of scientific literature. A single biological fact can be expressed in numerous ways, involving different parts of speech, active or passive voices, and varying levels of modality or background information. This variation makes it difficult for simple keyword or rule-based systems to capture the full scope of published knowledge.

Linguistic variations of biological interactions

Methods

The authors utilize Distant Supervision to automate knowledge extraction. This approach involves:

  1. Heuristic Labeling: Using the Pathway Interaction Database (PID) as a "cheat sheet," the system identifies protein pairs known to interact and searches PubMed for sentences containing both. It hypothesizes that these sentences describe the known interaction.
  2. Noise Reduction (MultiR): To handle sentences that mention proteins without describing an interaction (e.g., "We studied A and B"), the authors employed MultiR, a latent variable model that distinguishes between relevant expressions and "noise."
  3. Feature Engineering: The model analyzes lexical features (words between proteins) and syntactic features (the grammatical dependency path connecting the proteins).

The system was tested on the GENIA Event Extraction Dataset and then applied to the entire PubMed corpus of 22 million abstracts, using the Literome tool for protein identification.

Results

The distant supervision approach demonstrated significant improvements over traditional baselines. On the GENIA benchmark, it achieved an F1 score of 25.6, vastly outperforming the rule-based system (9.4) and nearing the performance of fully supervised models (33.2). This suggests that distant supervision can effectively bridge the gap created by the lack of human-labeled data.

Performance comparison of extraction systems

Key findings include:

  • Scale and Discovery: The system extracted 1.5 million interactions from PubMed, identifying over 210,000 unique relation triples—an order of magnitude larger than the training database.
  • Precision: Manual validation showed an end-to-end precision of 25%, rising to 31% when excluding protein tagging errors.
  • Cancer Specificity: By cross-referencing results with Medical Subject Headings (MeSH), the researchers identified genes central to specific cancers, such as BRCA1 in breast cancer and KRAS in lung cancer, creating a research landscape heatmap.

Discussion and Limitations

While successful, the system currently focuses on binary relations (Protein A -> Protein B), whereas biological interactions are often n-ary (complex groups). Additionally, approximately 20% of errors stemmed from Named Entity Recognition (tagging) issues. Future work will address complex relations, linguistic nuances like negation or hedging, and the implementation of "eCuration" loops where human experts verify AI-generated results to further refine the model.

Summary updated

Abstract

Biological pathways are central to understanding complex diseases such as cancer. The majority of this knowledge is scattered in the vast and rapidly growing research literature. To automate knowledge extraction, machine learning approaches typically require annotated examples, which are expensive and time-consuming to acquire. Recently, there has been increasing interest in leveraging databases for distant supervision in knowledge extraction, but existing applications focus almost exclusively on newswire domains. In this paper, we present the first attempt to formulate the distant supervision problem for pathway extraction and apply a state-of-the-art method to extracting pathway interactions from PubMed abstracts. Experiments show that distant supervision can effectively compensate for the lack of annotation, attaining an accuracy approaching supervised results. From 22 million PubMed abstracts, we extracted 1.5 million pathway interactions at a precision of 25%. More than 10% of interactions are mentioned in the context of one or more cancer types, analysis of which yields interesting insights.
BibTeX
@inproceedings{poon2015distant,
  author = {Hoifung Poon and Kristina Toutanova and Chris Quirk},
  title = {{Distant Supervision for Cancer Pathway Extraction from Text}},
  booktitle = {Pacific Symposium on Biocomputing},
  year = {2015},
  url = {https://openreview.net/forum?id=dgSOwKd1HYx},
}
Figure