Text mining facilitates database curation - extraction of mutation-disease associations from Bio-medical literature

Ravikumar Komandur Elayavilli, Kavishwar B. Wagholikar, Dingcheng Li, Jean-Pierre A. Kocher, Hongfang LiuBMC Bioinformatics 2015 · DOI

TL;DR

This paper introduces "MutD," a text mining system designed to automatically extract associations between protein mutations and diseases from biomedical literature. By incorporating discourse-level analysis, MutD significantly improves upon existing methods, achieving a notable F-measure and demonstrating its potential to accelerate the laborious process of biomedical database curation.

What Problem Does This Paper Address?

The rapid advancement of genetic sequencing technologies, particularly in the field of individualized medicine, has created an urgent need to catalog and interpret genetic variants and their associated diseases. While numerous biomedical databases exist, a significant bottleneck is the manual curation of information from scientific literature. This process is time-consuming, and there's a considerable lag between publication and data abstraction. Existing text mining tools often focus on sentence-level extraction and are evaluated against text-based annotations, which may not fully reflect the complexity and inferential reasoning required for database curation. This paper addresses the challenge of extracting protein mutation-disease associations from biomedical literature in a way that is practical for database curation, going beyond simple sentence-level co-occurrence.

Methods

The core of this work is the development of the "MutD" text mining system, built on the Apache UIMA framework. MutD employs a multi-stage approach:

  • Information Retrieval: Abstracts are retrieved from PubMed using specific queries.
  • Preprocessing: Standard NLP techniques like sentence splitting and tokenization are applied.
  • Term Identification: Named entities (proteins, mutations, diseases) are identified using a combination of existing tools like PubTator, BioTagger-GM, and MutationFinder. These entities are then normalized to database identifiers (e.g., UniProt IDs, OMIM IDs).
  • Relation Extraction: This is where MutD introduces its novelty. It moves beyond sentence-level analysis by incorporating:
  • Discourse-level analysis: This allows MutD to link information across multiple sentences.
  • Dependency Graph Traversal: Sentences are parsed into dependency graphs, and relationships are inferred by traversing these graphs.
  • Cross-sentence Linking: Graphs from different sentences are linked based on shared entities, anaphora resolution (pronoun references), and "trigger words" that connect concepts across sentences.

Datasets

The study utilizes publicly available resources for its tools and evaluation:

  • UniProtKB (SwissProt section): Used to extract curated protein mutation-disease associations to create a gold standard dataset.
  • PubMed Abstracts: The source of text data for extraction.
  • MEDLINE: The database from which abstracts are retrieved.
  • Ontologies/Vocabularies: UMLS, BioThesaurus, MEDIC, MeSH, OMIM, and UniProtKB are used for entity normalization.

The authors created a benchmark dataset by extracting gold standard associations from UniProtKB records, specifically focusing on abstracts cited as evidence for protein-mutation-disease relationships. This dataset was split into development (576 abstracts) and test (144 abstracts) sets.

Key Insights and Ideas

  • Discourse Matters: The most significant insight is that analyzing text beyond individual sentences is crucial for accurately extracting complex relationships like protein-mutation-disease associations. Simply looking at co-occurrence within a sentence is insufficient.
  • Database Curation is Different: Evaluating text mining systems against database-curated gold standards, rather than just text-based annotations, provides a more realistic measure of their "practical utility" for real-world curation tasks.
  • Human Inference is Still Key (for now): The paper highlights that certain aspects of database curation, particularly inferring context like signal peptide regions, still require human expertise and are challenging for current text mining systems.
  • Error Analysis Reveals Deeper Issues: The detailed error analysis shows that a significant portion of "missed" associations might actually be valid findings not yet captured by database curators, suggesting the system is finding novel information.

Key Results

  • MutD achieved an F-measure of 64.3% for reconstructing protein mutation-disease associations when evaluated against curated database records.
  • The discourse-level analysis component of MutD contributed to a gain of over 10% in F-measure compared to sentence-level association extraction.
  • The most advanced system variant (S5), which incorporated discourse-level analysis, anaphora resolution, and trigger words, achieved the best F-measure of 66.7% on the development set and 64.3% on the test set.
  • An error analysis revealed that 23% of precision errors were due to associations not yet present in UniProtKB, and 68% of recall errors were due to the absence of associated disease entities in the abstract.
  • After adjusting for these "defects" in the curated database, the revised F-measure of MutD reached 81.5%.

Limitations and Future Work

  • Scope: The study focused on protein point mutations and did not cover other variant types like deletions or additions.
  • Full Text Processing: The current system processes abstracts. Extending it to full-text articles is a logical next step, though it will introduce challenges in managing irrelevant information.
  • Expert Inference: The paper acknowledges that the "expert inference challenge" (e.g., interpreting signal peptide regions) cannot be fully addressed by text mining alone.
  • Novelty: The system identified associations not yet present in UniProtKB, suggesting potential for discovering new knowledge.
  • Future Directions: The authors plan to release web and RESTAPI versions of MutD, extend it to extract drug-mutation-disease ternary relations, and integrate it with other knowledge sources to support individualized medicine.

Summary updated

BibTeX
@article{elayavilli2015text,
  author = {Ravikumar Komandur Elayavilli and Kavishwar B. Wagholikar and Dingcheng Li and Jean-Pierre A. Kocher and Hongfang Liu},
  title = {{Text mining facilitates database curation - extraction of mutation-disease associations from Bio-medical literature}},
  journal = {BMC Bioinformatics},
  year = {2015},
  doi = {10.1186/s12859-015-0609-x},
}
Figure