High-precision biological event extraction with a concept recognizer

K. Bretonnel Cohen, Karin Verspoor, Helen L. Johnson, Christophe Roeder, Philip V. Ogren, William A. Baumgartner Jr., Elizabeth K. White, Lawrence HunterNAACL 2009 · OpenReview

Abstract: K. Bretonnel Cohen, Karin Verspoor, Helen Johnson, Chris Roeder, Philip Ogren, William Baumgartner, Elizabeth White, Lawrence Hunter. Proceedings of the BioNLP 2009 Workshop Companion Volume for Shared Task.

2009.

Introduction

In the field of biomedical natural language processing (BioNLP), the focus has shifted from simple named entity recognition to the extraction of complex biological events. These events, such as protein binding or gene expression, involve specific participants and spatial contexts. The BioNLP’09 Shared Task challenged researchers to automate this extraction process. This paper presents a system based on the OpenDMAP ontology-driven semantic parser, which prioritizes high precision through the use of manually crafted linguistic rules and ontological constraints.

Methods

The authors approached event extraction as a concept recognition task. Instead of relying on deep machine learning models, they utilized OpenDMAP, a parser that matches text against patterns defined by ontological classes.

Central to this approach was the integration of community-consensus ontologies, including the Gene Ontology (GO), Sequence Ontology (SO), and Cell Type Ontology (CTO). These ontologies were used to strictly constrain the valid arguments for each event type. For example, a "binding" event is restricted to occurring at specific sites like a protein domain or DNA segment.

Ontological constraints for biological event types

The system pipeline included:

  • Named Entity Recognition (NER): Proteins were tagged using LingPipe, while other semantic classes were identified via UIMA ConceptMapper.
  • Manual Pattern Development: The team developed a "semantic grammar" based on trigger words, with a heavy focus on nominalizations (e.g., "phosphorylation"), which were found to be the most common way events are expressed in the literature.
  • Coordination Handling: An OpenNLP constituent parser was integrated to correctly extract multiple events from coordinated lists of proteins.

Results

The system was evaluated against the BioNLP’09 Shared Task dataset. It achieved the highest precision among all participating teams for the primary tasks, confirming that ontologically constrained rule-based systems can be exceptionally accurate.

Official BioNLP'09 Shared Task results showing high precision

As shown in the official results, the system achieved 71.81% precision for Task 1 (Event Detection) and 70.97% precision for Task 2 (Event Arguments). However, this focus on precision resulted in low recall (approximately 13%), leading to an overall F-measure of 22.66% for Task 1. Post-submission bug fixes later improved the Task 1 F-measure to 27.62% by raising recall to 17.38%.

Discussion and Limitations

The primary advantage of this approach is its reliability; the high precision ensures that extracted events are highly likely to be biologically valid. The authors argue that in a redundant scientific literature, a high-precision/low-recall system is valuable for building high-quality knowledge bases, as important facts are likely to be stated simply enough to be captured by the rules at least once.

The main limitation is the "recall bottleneck." The system struggled with:

  1. Intervening Material: Unexpected words between a trigger and its argument often broke the rigid patterns.
  2. Anaphora and Coreference: The system could not resolve pronouns or indirect references.
  3. Complex Syntax: Structures like appositives often confused the pattern matcher.

Future work will involve incorporating robust syntactic analysis, such as dependency parsing, to handle more complex sentence structures and improve the system's overall recall while maintaining its high precision standards.

Summary updated

BibTeX
@inproceedings{cohen2009high,
  author = {K. Bretonnel Cohen and Karin Verspoor and Helen L. Johnson and Christophe Roeder and Philip V. Ogren and William A. Baumgartner Jr. and Elizabeth K. White and Lawrence Hunter},
  title = {{High-precision biological event extraction with a concept recognizer}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2009},
  url = {https://openreview.net/forum?id=mU0g4L476s0},
}
Figure