Extending NegEx with Kernel Methods for Negation Detection in Clinical Text

Chaitanya Shivade, Marie-Catherine de Marneffe, Eric Fosler‐Lussier, Albert M. Lai2015 · DOI

Abstract: Chaitanya Shivade, Marie-Catherine de Marneffe, Eric Fosler-Lussier, Albert M. Lai. Proceedings of the Second Workshop on Extra-Propositional Aspects of Meaning in Computational Semantics (ExProM 2015).

2015.

Introduction

In clinical informatics, accurately identifying the presence or absence of medical concepts is a fundamental task. A search for "malignancy" in an electronic medical record (EMR) might return "The scan showed no signs of malignancy," where a simple keyword match would result in a false positive. For years, the rule-based NegEx system has been the industry standard, using regular expressions to identify "trigger" phrases like "no signs of." However, NegEx is inherently brittle, struggling to generalize across different institutions or document types due to its reliance on hand-crafted rules. This paper explores evolving NegEx into a more robust machine learning model using kernel methods.

Methodology

The authors propose using Support Vector Machines (SVMs) with various Kernel Methods to bridge the gap between rigid rules and flexible machine learning. A kernel acts as a similarity function, allowing the model to compare sentences in a high-dimensional space. Key approaches include:

  1. NegEx Features Kernel (NF): Encodes NegEx's internal logic (rules and conjunctions) as binary features.
  2. Augmented Bag of Words Kernel (ABoW): Combines standard word-presence features with the final decision from the original NegEx system as an additional feature.
  3. Composite Kernels: Summation of different kernels to leverage both rule-based logic and raw text patterns.
  4. Dependency Tree Kernels: Utilizes grammatical structure, though this proved less effective in the often-fragmented context of clinical notes.
  5. Semi-supervised Learning: Uses NegEx to "pseudo-label" large unannotated datasets for training new models.

Datasets

The study utilized two primary clinical text sources:

  • NegCorp: Sentences from the University of Pittsburgh Medical Center, used to validate the original NegEx.
  • i2b2 2010 Challenge Data: A benchmark dataset including notes from Partners Healthcare (PH) and Beth Israel Deaconess (BID).

Results and Discussion

The ABoW (Augmented Bag of Words) kernel emerged as the superior method. By treating the rule-based decision as a "hint" rather than a law, the model learned to contextualize NegEx's output with surrounding words.

Performance comparison of different kernel methods across datasets.

As shown in the performance comparison, the ABoW kernel consistently outperformed the NegEx baseline across all datasets, reaching an F1-score of 97.0 on NegCorp and significantly improving results on the BID and PH subsets. The composite ABoW+NF kernel provided even further marginal gains.

The Recall Rescue

A major advantage of these kernel methods is the significant boost in recall. Rule-based systems often miss negations they haven't been explicitly programmed to find. The ABoW model picks up on subtle linguistic cues that fall outside the original NegEx rulebook.

Precision, Recall, and F1 scores for NegEx vs. ABoW models.

The breakdown of precision and recall highlights that while precision remains high and stable, the recall for ABoW models trained on clinical data (PH, BID, i2b2Train) is substantially higher than the NegEx baseline, leading to better overall F1-scores. For instance, training on the full i2b2 training set improved recall from 79.9% to 90.0%.

Semi-Supervised Learning

The authors also demonstrated that a model trained on "pseudo-labels" generated by NegEx (ABoW$_{ss}$) could outperform NegEx itself, achieving an F1 of 85.7 compared to the baseline's 84.5. This suggests that imperfect legacy systems can be used to bootstrap more powerful modern classifiers without requiring extensive new human annotation.

Conclusion

This research demonstrates that integrating rule-based logic into machine learning frameworks via kernel methods significantly enhances negation detection in clinical text. The ABoW approach provides a robust, high-recall alternative to traditional systems, proving particularly effective in cross-institutional settings. Future work may focus on refining structural analysis using domain-specific medical ontologies like UMLS to better handle the ungrammatical nature of clinical notes.

Summary updated

BibTeX
@misc{shivade2015extending,
  author = {Chaitanya Shivade and Marie-Catherine de Marneffe and Eric Fosler‐Lussier and Albert M. Lai},
  title = {{Extending NegEx with Kernel Methods for Negation Detection in Clinical Text}},
  year = {2015},
  doi = {10.3115/v1/w15-1305},
}
Figure