OpenDMAP: An open source, ontology-driven concept analysis engine, with applications to capturing knowledge regarding protein transport, protein interactions and cell-type-specific gene expression

Lawrence Hunter, Zhiyong Lu, James Firby, William A. Baumgartner Jr., Helen L. Johnson, Philip V. Ogren, K. Bretonnel Cohen

TL;DR

This paper introduces OpenDMAP, an open-source information extraction system that leverages ontologies to understand biomedical text. It significantly improves the accuracy of extracting complex biological relationships, like protein transport and interactions, by integrating diverse NLP tools and a flexible pattern language.

What Problem Does This Paper Address?

The explosion of biomedical literature makes it impossible for researchers to manually keep up with new discoveries. Extracting specific, structured knowledge from this vast amount of text is crucial for building comprehensive biological databases and advancing research. However, existing information extraction (IE) systems often struggle with the complexity and nuance of biological language, particularly when dealing with relationships involving more than two entities or when needing to ground extracted information in a formal model of biological reality. This paper aims to address these limitations by developing a more robust and flexible IE system.

Methods

OpenDMAP is built upon the Direct Memory Access Parsing (DMAP) paradigm and introduces several key innovations:

  • Ontology-Driven: It uses Protégé to model biological concepts and relationships, allowing for a structured and consistent knowledge representation. This means the system's output is always grounded in a formal ontology, unlike systems that rely solely on text substrings.
  • Integrated NLP Pipeline: OpenDMAP leverages the Unstructured Information Management Architecture (UIMA) to integrate various off-the-shelf NLP tools (e.g., tokenizers, parsers, named entity recognizers). This allows for flexible experimentation and combination of different linguistic processing approaches.
  • Expanded Pattern Language: The system uses a flexible pattern language that can mix syntactic and semantic elements, and crucially, allows for variable ordering. This enables the specification of complex relationships and handles variations in how these relationships are expressed in text. It also supports patterns that go beyond simple binary relationships.

Datasets

The paper evaluates OpenDMAP on three specific information extraction tasks:

  • Protein Transport: Evaluated on GeneRIFs (short, curated descriptions of gene functions).
  • Protein-Protein Interactions: Evaluated on abstracts from MEDLINE and also as part of the BioCreative II protein-protein interaction (PPI) shared task, which used full-text articles.
  • Gene Expression: Evaluated on GeneRIFs.

The system was also run over all abstracts in MEDLINE to demonstrate its scalability.

Key Insights and Ideas

  • Ontologies are Key for Robust IE: Grounding information extraction in formal ontologies provides a crucial layer of semantic consistency and interpretability that is often missing in other IE systems.
  • Flexibility in Pattern Matching is Essential: Biological language is highly variable. OpenDMAP's pattern language, which allows for mixing syntactic and semantic elements and variable ordering, is critical for capturing these variations effectively.
  • Beyond Binary Relationships: Many important biological relationships are not simple pairwise interactions. OpenDMAP's ability to handle more complex, n-ary relationships is a significant advancement.
  • Integration of NLP Tools is Powerful: By using UIMA, OpenDMAP can easily incorporate and leverage the strengths of various existing NLP tools, creating a more powerful and adaptable system.

Key Results

  • Improved Performance: OpenDMAP achieved competitive results, particularly in the BioCreative PPI task, where it scored 10% higher than the next best system in F-measure and had significantly higher precision.
  • Large-Scale Extraction: The system was applied to all MEDLINE abstracts, yielding a substantial number of extracted instances: 72,460 for protein transport, 265,795 for protein-protein interactions, and 176,153 for gene expression.
  • Broad Coverage: The system extracted information from a significant portion of biomedical journals indexed in MEDLINE, demonstrating its broad applicability.
  • Manual Evaluation Insights: Manual evaluations highlighted that while OpenDMAP performs well, errors in gene/protein name recognition and syntactic parsing are significant sources of inaccuracy.

Limitations and Future Work

  • Error Sources: The authors acknowledge that errors in gene/protein name recognition and syntactic analysis are major contributors to false positives.
  • Parser Performance: The Stanford parser, while providing valuable syntactic information, was too slow for large-scale processing of the entire MEDLINE database.
  • Future Directions: The authors suggest that incorporating improved gene/protein recognition systems (like those used in BioCreative) and specialized biomedical parsers could significantly enhance OpenDMAP's accuracy. They also note that addressing issues like anaphora resolution, negation, and improving the handling of coordinated clauses and conjunctions are areas for future research. The development of tools to integrate IE results into curatorial workflows is also an ongoing effort.

Summary updated

Abstract

Information extraction (IE) efforts are widely acknowledged to be important in harnessing the rapid advance of biomedical knowledge, particularly in areas where important factual information is published in a diverse literature. Here we report on the design, implementation and several evaluations of OpenDMAP, an ontology-driven, integrated concept analysis system. It significantly advances the state of the art in information extraction by leveraging knowledge in ontological resources, integrating diverse text processing applications, and using an expanded pattern language that allows the mixing of syntactic and semantic elements and variable ordering. OpenDMAP information extraction systems were produced for extracting protein transport assertions (transport), protein-protein interaction assertions (interaction) and assertions that a gene is expressed in a cell type (expression). Evaluations were performed on each system, resulting in F-scores ranging from .26 – .72 (precision .39 – .85, recall .16 – .85). Additionally, each of these systems was run over all abstracts in MEDLINE, producing a total of 72,460 transport instances, 265,795 interaction instances and 176,153 expression instances. OpenDMAP advances the performance standards for extracting protein-protein interaction predications from the full texts of biomedical research articles. Furthermore, this level of performance appears to generalize to other information extraction tasks, including extracting information about predicates of more than two arguments. The output of the information extraction system is always constructed from elements of an ontology, ensuring that the knowledge representation is grounded with respect to a carefully constructed model of reality. The results of these efforts can be used to increase the efficiency of manual curation efforts and to provide additional features in systems that integrate multiple sources for information extraction. The open source OpenDMAP code library is freely available at http://bionlp.sourceforge.net/
BibTeX
@article{hunter2008opendmap,
  author = {Lawrence Hunter and Zhiyong Lu and James Firby and William A. Baumgartner Jr. and Helen L. Johnson and Philip V. Ogren and K. Bretonnel Cohen},
  title = {{OpenDMAP: An open source, ontology-driven concept analysis engine, with applications to capturing knowledge regarding protein transport, protein interactions and cell-type-specific gene expression}},
  journal = {BMC Bioinformatics},
  year = {2008},
  doi = {10.1186/1471-2105-9-78},
}
Figure