Large-scale Information Extraction for Assisted Curation of the Biomedical Literature

Fabio Rinaldi, Lenz Furrer, Simon ClematideIT@LIA@AI*IA 2015 · OpenReview

Introduction

The biomedical community is currently grappling with a massive "data deluge." With over 23 million references in PubMed and approximately two new publications added every minute, it has become humanly impossible for researchers and curators to keep pace with the literature. Vital knowledge regarding the interactions between proteins, genes, and drugs is often "trapped" within unstructured text. While professional curators manually update databases like UniProt or BioGrid, the process is inherently slow and resource-intensive. To address this, researchers from the University of Zurich developed the OntoGene pipeline, a text-mining system designed to extract protein-protein interactions (PPIs) and facilitate semi-automated curation.

The OntoGene Pipeline

The OntoGene pipeline employs a multi-stage process to transform unstructured text into structured, actionable data:

  1. Entity Recognition & Disambiguation: The system identifies potential terms using terminology from life science databases. It utilizes an "approximate matching" approach to ensure high recall, followed by a machine-learning filter to resolve ambiguities—such as determining which specific "p53" protein a paper refers to.
  2. Relation Extraction: Once entities are identified, the system generates all possible pairwise combinations of detected entities within a document.
  3. Scoring Mechanism: Interactions are ranked using a combination of a "Concept Score" (likelihood of a term-identifier pair appearing in curated databases) and a "Sentence Score" (a Naïve Bayes model analyzing the linguistic context). The system also calculates a "collectionScore" based on global evidence across the entire literature set.
Ranked protein-protein interaction results based on global collection scores.

Human-in-the-Loop: The ODIN Interface

A core philosophy of the OntoGene project is that automated systems should assist, rather than replace, human experts. The OntoGene Document Inspector (ODIN) is a web-based interface that allows curators to review the system's findings. ODIN presents original text snippets with highlighted entities, displays the system's confidence scores, and enables curators to confirm or reject interactions with minimal effort.

The ODIN interface showing highlighted protein entities and interaction evidence within document snippets.

Key Insights and Results

The system prioritizes ranking quality over raw precision. In assisted curation, the goal is to present the most likely correct results at the top of the list. Even if overall precision is modest, a high "precision at k" (measured by the TAP-10 score) significantly boosts curator efficiency.

  • Entity Recognition: The system achieved a recall of 70.6%. While initial precision was 11.6%, simple exclusion rules for frequent false positives improved this to 14.6%.
  • Relation Extraction: The pipeline reached a TAP-10 score of 0.229.
  • Practical Application: The system has been used to analyze the entirety of PubMed, creating a searchable PPI database that is currently integrated into projects like RegulonDB to improve curation efficiency.

Conclusion and Future Work

While the OntoGene pipeline demonstrates the power of large-scale information extraction, challenges remain. Current evaluations focus on PubMed abstracts, yet critical evidence often resides in full-text articles, including methods and image captions. Future developments will focus on detecting specific interaction types (e.g., phosphorylation) and directionality. By keeping the "human in the loop," OntoGene provides a scalable solution to the biomedical data deluge.

Summary updated

BibTeX
@inproceedings{rinaldi2015large,
  author = {Fabio Rinaldi and Lenz Furrer and Simon Clematide},
  title = {{Large-scale Information Extraction for Assisted Curation of the Biomedical Literature}},
  booktitle = {IT@LIA@AI*IA},
  year = {2015},
  url = {https://openreview.net/forum?id=oZEVOURLNAY},
}
Figure