Attention-Based Deep Learning System for Negation and Assertion Detection in Clinical Notes

Long ChenInternational Journal of Artificial Intelligence & Applications 2019 · DOI

TL;DR

This paper introduces an "attention-based deep learning system" (Att-BiLSTM) for automatically identifying whether a medical finding in clinical notes is present, absent (negated), uncertain, or hypothetical. The system achieves state-of-the-art performance without relying on external knowledge bases, demonstrating the power of deep learning for nuanced clinical text analysis.

What Problem Does This Paper Address?

Clinical notes, a rich source of patient information in Electronic Health Records (EHRs), are unstructured text. Extracting medical concepts from these notes is only the first step. Crucially, the meaning of these concepts is heavily influenced by modifiers like negation ("no evidence of chest pain"), uncertainty ("possible infection"), or hypotheticals ("would suggest a fracture"). Misinterpreting these assertions can lead to incorrect diagnoses, flawed medical research, and poor decision-making. Existing methods often rely on manually crafted rules or extensive domain knowledge, making them difficult to adapt to new contexts. This paper aims to develop a high-performance, knowledge-lean system for robust assertion detection in clinical notes.

Methods

The core of the proposed system is an Attention-based Bidirectional Long Short-Term Memory (Att-BiLSTM) network. Here's a breakdown:

  • Bidirectional LSTM (BiLSTM): This is a standard deep learning architecture for sequence processing. It processes the input text in both forward and backward directions, allowing it to capture context from words both before and after a target medical concept.
  • Word Embeddings: Words are converted into dense numerical vectors (embeddings). The paper explores different pre-trained embeddings (randomly initialized, MIMIC, and PubMed+) to represent words.
  • Attention Mechanism: This is the novel component. Instead of treating all words in a sentence equally, the attention mechanism allows the model to "focus" on the most relevant words when determining the assertion status of a target concept. This helps the model pinpoint negation cues or other modifiers more effectively.
  • Knowledge-Poor Approach: Unlike many previous systems that require lexicons, ontologies, or manually engineered features, this deep learning model learns directly from the data. This makes it more adaptable and easier to extend to different clinical domains.

Datasets

The researchers utilized publicly available datasets for their experiments:

  • i2b2/VA Challenge Datasets: These are the primary datasets used. They include discharge summaries and progress notes from various medical centers. The annotations cover six assertion categories: Present, Absent, Hypothetical, Possible, Conditional, and Associated With Someone Else (AWSE).
  • i2b2-BID/PH: A subset of the i2b2 data.
  • i2b2-UPMC: The original 2010 i2b2 test dataset.
  • NegEx-Corp Dataset: This dataset, derived from the NegEx rule-based system's source code, contains clinical conditions and their sentences, annotated as either negated or affirmed. It's primarily used for evaluating negation detection.

The paper constructs three specific datasets from these sources: NegExCorp, i2b2-BID/PH, and i2b2-UPMC.

Key Insights and Ideas

  • Attention is Crucial for Clinical Nuance: The attention mechanism proves vital for pinpointing the specific words that modify the assertion of a clinical concept. This is especially important in clinical text where subtle phrasing can drastically alter meaning.
  • Deep Learning Can Be Knowledge-Lean and High-Performing: The study demonstrates that a deep learning approach, without extensive feature engineering or domain knowledge, can achieve performance comparable to or exceeding state-of-the-art systems. This opens doors for more generalizable NLP solutions in healthcare.
  • Pre-trained Embeddings Matter: Using pre-trained word embeddings, particularly those trained on large biomedical corpora like PubMed+, significantly boosts the model's performance compared to random initialization or embeddings trained on more general data.
  • Generalizability Across Datasets: The Att-BiLSTM model shows good generalizability, performing well even when trained on one dataset and tested on another, suggesting its potential for broad application.

Key Results

  • The Att-BiLSTM system achieved an overall micro-F1 score of 0.922 for the assertion detection task on the i2b2-UPMC dataset. This is comparable to the top-performing systems in the 2010 i2b2/VA challenge.
  • For negation detection (Neg1 and Neg2 metrics), the proposed model outperformed the NegEx rule-based system on the i2b2-UPMC dataset, achieving F1 scores of 0.913 (Neg1) and 0.913 (Neg2) compared to NegEx's 0.845 and 0.929 respectively.
  • Using PubMed+ word embeddings yielded the best results across various assertion classes, improving F1 scores compared to random or MIMIC embeddings.
  • Cross-dataset evaluations showed that models trained on larger datasets (like i2b2-UPMC) tended to perform slightly better, indicating that more data can further enhance performance.

Limitations and Future Work

  • Dataset Size: While the system performs well, the authors note that larger datasets could potentially lead to even better performance, as suggested by the comparison between models trained on i2b2-BID/PH and i2b2-UPMC.
  • Generalizability to Other Domains: Although the knowledge-lean approach suggests good generalizability, further testing on diverse clinical note types or even non-clinical domains would be beneficial.
  • Specific Assertion Types: While the system handles multiple assertion types, the paper focuses heavily on negation. Future work could explore more in-depth analysis and improvement for less common assertion categories.
  • Computational Cost: Deep learning models, especially LSTMs, can be computationally intensive. While not explicitly detailed as a limitation, optimizing for efficiency in real-world clinical settings would be a practical next step.

Summary updated

Abstract

Natural language processing (NLP) has been recently used to extract clinical information from free text in Electronic Health Record (EHR). In clinical NLP one challenge is that the meaning of clinical entities is heavily affected by assertion modifiers such as negation, uncertain, hypothetical, experiencer and so on. Incorrect assertion assignment could cause inaccurate diagnosis of patients’ condition or negatively influence following study like disease modelling. Thus, high-performance clinical NLP systems which can automatically detect negation and other assertion status of given target medical findings (e.g. disease, symptom) in clinical context are highly demanded. Here in this work, we propose a deep-learning system based on word embedding and Attention-based Bidirectional Long Short-Term Memory networks (AttBiLSTM) for assertion detection in clinical notes. Unlike previous state-of-art methods which require knowledge input, our system is a knowledge poor machine learning system and can be easily extended or transferred to other domains. The evaluation of our system on public benchmarking corpora demonstrates that a knowledge poor deep-learning system can also achieve high performance for detecting negation and assertions comparing to state-of-the-art systems.
BibTeX
@article{chen2019attention,
  author = {Long Chen},
  title = {{Attention-Based Deep Learning System for Negation and Assertion Detection in Clinical Notes}},
  journal = {International Journal of Artificial Intelligence \& Applications},
  year = {2019},
  doi = {10.5121/ijaia.2019.10101},
}
Figure