Exploiting Context for Biomedical Entity Recognition: From Syntax to the Web

Jenny Rose Finkel, Shipra Dingare, Huy Nguyen, Malvina Nissim, Christopher D. Manning, Gail Sinclair

Abstract: Jenny Finkel, Shipra Dingare, Huy Nguyen, Malvina Nissim, Christopher Manning, Gail Sinclair. Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP).

2004.

Introduction

Named Entity Recognition (NER) in the biomedical domain is notoriously difficult compared to standard newswire text. Biomedical language is highly complex, idiosyncratic, and filled with acronyms and systematic polysemies, where the same word can mean different things depending on context. Furthermore, internal word cues—like capitalization rules that easily identify names in regular English—are often inconsistent or lacking.

This paper addresses the Coling 2004 NLPBA/BioNLP shared task, which requires systems to accurately identify and classify five specific entities—protein, RNA, DNA, cell line, and cell type—within the GENIA corpus of MEDLINE abstracts. Because these entities lack strong internal cues, the challenge is to creatively leverage surrounding context and external knowledge to classify them correctly.

Methods

The core of the system is a Maximum Entropy Markov Model (MEMM). To overcome the lack of internal word cues, the authors engineered a diverse set of features, moving from surface-level syntax to external web data.

Table 1: Feature set used in the MEMM model, including word features, POS tags, and word shapes.

The model utilizes several layers of information:

  • Local & Syntactic Features: The model examines word shapes (capitalization, numerals, Greek letters), prefixes/suffixes, part-of-speech tags (using a retrained TnT tagger), and parenthesis-matching to identify multi-word entities.
  • Deep Syntax: Text was processed using the unlexicalized Stanford Parser to extract the head and governor of noun phrases, providing structural context.
  • External Gazetteers & Frequency: The researchers used a massive gazetteer of 1.7 million gene names. To combat ambiguity, they cross-referenced word frequencies using the British National Corpus (BNC). Rare words in the BNC that appear in a gene gazetteer are significantly more likely to be actual genes in context.
  • Web Querying: For rare words (appearing fewer than 10 times in the BNC), they queried the Google API with indicative patterns (e.g., "X mRNA" or "X ligation") to gather contextual clues based on hit counts.
  • Document-Level Context: Each abstract was tagged twice to leverage instances where an ambiguous token might appear in a more obvious context elsewhere in the same document.

Key Results

The system achieved an overall F-score of 70.06% (71.62% Precision, 68.56% Recall) on the evaluation data. Performance varied significantly by entity type: Proteins were the easiest to identify (72.67% F-score), while cell lines were the hardest (52.40% F-score).

Table 2: Performance metrics for entity recognition under different boundary matching criteria.

A key insight from the results is the difficulty of boundary detection. Relaxing the evaluation metric to allow for just the correct left boundary or right boundary increased the overall F-score to 74.07% and 77.91%, respectively. This demonstrates that determining the exact start and end of multi-word entities is a major source of error.

Discussion and Limitations

The authors argue that comparing biomedical NER to newswire NER (which often hits 90%+ accuracy) is flawed. Biomedical NER uses stricter "exact match" evaluation metrics and the ground truth data itself suffers from lower inter-annotator agreement due to the inherent "fuzziness" of biology.

Several limitations were noted:

  • Adjacent Entities: By collapsing the B- and I- tags to save memory, the system automatically merges adjacent entities of the same class into a single entity, causing boundary errors.
  • Annotation Inconsistencies: Significant inconsistencies exist in the GENIA ground-truth data. For example, preceding adjectives (like "human" in "human monocytes") are included in the entity boundary inconsistently, confusing the model.

Future work will require more sophisticated use of ontologies, deeper semantic processing, and a better understanding of inter-annotator agreement limits in the biomedical domain.

Summary updated

BibTeX
@inproceedings{finkel2004exploiting,
  author = {Jenny Rose Finkel and Shipra Dingare and Huy Nguyen and Malvina Nissim and Christopher D. Manning and Gail Sinclair},
  title = {{Exploiting Context for Biomedical Entity Recognition: From Syntax to the Web}},
  booktitle = {NLPBA/BioNLP},
  year = {2004},
  url = {https://openreview.net/forum?id=hRDVx5XmwN},
}
Figure