Context-aware Medication Event Extraction from Unstructured Text

Noushin Salek Faramarzi, Meet Patel, Sai Harika Bandarupally, Ritwik BanerjeeACL 2023

TL;DR

This paper tackles the challenge of accurately extracting medication-related events and their contextual attributes from unstructured clinical notes. By leveraging the Contextualized Medication Event Dataset (CMED) and employing advanced NLP techniques like domain-specific pre-trained language models and data augmentation, the authors achieve improved performance in this critical task for healthcare.

What Problem Does This Paper Address?

Accurate medication history is fundamental for high-quality medical care, enabling healthcare professionals to assess treatment effectiveness, identify potential problems, and plan future interventions. However, crucial information about medication changes is often buried within unstructured clinical narratives, making it difficult to extract systematically. Existing structured data sources frequently lack this detail. The challenge lies in not just identifying medication mentions but also understanding the context surrounding them, such as the type of change (start, stop, increase), the actor responsible, negation, temporality, and certainty. This complexity makes it difficult for automated systems to capture the full picture of a patient's medication journey.

Methods

The authors employ a multi-stage approach to medication event extraction.

  • Medication Mention Extraction: They frame this as a sequence tagging task, using BIO (Beginning, Inside, Outside) labeling. They leverage pre-trained Transformer models, specifically domain-specific ones like BioBERT and Bio+Clinical BERT, fine-tuning them for this task. Data augmentation with the DDI (Drug-Drug Interaction) corpus is used to improve performance.
  • Abbreviation Resolution: To handle variations in clinical text, they resolve common abbreviations for terms like "continue" and "discontinue" to their full forms, improving the robustness of downstream processing.
  • Coreference Resolution: They utilize AllenNLP's coreference resolution model to link pronouns and other referring expressions to their antecedents, enhancing the clarity of the text and aiding in actor classification.
  • Syntactic Dependency-Based Sentence Extraction: A novel sentence extraction method is developed. It parses sentences using spaCy, identifies coordinated conjunction phrases, and extracts the longest continuous sequences of dependent words. This aims to isolate the most relevant context for medication events.
  • Sentence Separation: To handle sentences with multiple medication mentions and event types, they split sentences into distinct clausal components, allowing for more precise event-to-drug association.
  • Event and Attribute Classification: For classifying the event type (Disposition, NoDisposition, Undetermined) and its associated attributes (action, actor, negation, certainty, temporality), they train classification models based on Transformer architectures (BERT, RoBERTa, XLNet, Bio+Clinical BERT). These models are trained on sentences containing the detected medication mention as context.
  • Negation Detection: They employ negspaCy, a Python library designed for detecting negation and related linguistic phenomena in clinical text, integrating it with their models.

Datasets

  • Contextualized Medication Event Dataset (CMED): This is the primary dataset used for training and evaluation. It comprises annotated clinical notes with medication mentions, classified into Disposition, NoDisposition, or Undetermined events. For Disposition events, five orthogonal dimensions of contextual information (action, actor, negation, certainty, temporality) are annotated.
  • DDI (Drug-Drug Interaction) Corpus: This corpus, containing sentences from biomedical literature discussing drug-drug interactions, was used for data augmentation to improve medication mention extraction.

Key Insights and Ideas

  • Importance of Context: The paper strongly emphasizes that simply extracting medication mentions is insufficient. Understanding the surrounding linguistic context is crucial for accurately identifying and classifying medication events and their attributes.
  • Data Preprocessing is Key: The authors highlight that sophisticated data preprocessing techniques (abbreviation resolution, coreference resolution, custom sentence extraction) significantly improve the performance of even advanced language models on clinical text.
  • Domain-Specific Models Shine: BioBERT and Bio+Clinical BERT consistently outperform general-purpose models like BERT, underscoring the value of pre-training on biomedical data for this task.
  • Data Augmentation Benefits: Augmenting the CMED training data with the DDI corpus led to performance improvements, particularly for medication mention extraction.
  • Challenges in Clinical Narratives: The paper points out inherent difficulties in clinical text, such as sentences with multiple drug mentions, ambiguous labeling, and information spanning across sentences, which pose significant challenges for automated systems.

Key Results

  • Medication Mention Extraction: BioBERT achieved the highest F1 scores (0.95 strict, 0.96 lenient) on the CMED test set. Augmenting with the DDI corpus showed further improvements.
  • Event and Attribute Classification:
    • For event classification, micro F1 scores ranged from 0.91 to 0.93. Bio+Clinical BERT achieved the highest F1 score (0.93).
  • For attribute classification, micro F1 scores ranged from 0.77 to 0.86. Bio+Clinical BERT performed best for certainty (0.86) and actor classification (0.94 precision).
  • Temporality and actor classification were noted as more challenging, showing lower performance across models.
  • Negation Detection: The system achieved a near-perfect accuracy of 0.98 for negation attribute classification, with precision, recall, and macro-average F1 of 0.82, 0.88, and 0.85, respectively, despite the very small proportion of negated events in the dataset.

Limitations and Future Work

  • Annotation Quality and Quantity: The paper notes that the limited number of negated samples and potential annotation errors in the CMED dataset (e.g., incorrect negation labels, ambiguous labeling for actor/temporality, misclassification of events) can negatively impact model performance.
  • Sentence-Level vs. Document-Level Context: While the custom sentence extraction aims to capture relevant context, the paper acknowledges that information about medication events can extend beyond a single sentence, sometimes requiring analysis of the entire document or even patient history. This is identified as a challenging area for future work.
  • Multi-Label Classification: The authors suggest that modifying the task from sentence classification to multi-label classification could help address cases where a sentence has multiple event annotations.
  • Handling Non-English Sentences: The presence of non-English sentences in the training set was noted as a potential issue impacting performance.
  • Undetermined Events: The difficulty in classifying "Undetermined" events, often found in assessment and plan sections, is highlighted as a complex task requiring broader contextual analysis.

Summary updated

BibTeX
@inproceedings{faramarzi2023context,
  author = {Noushin Salek Faramarzi and Meet Patel and Sai Harika Bandarupally and Ritwik Banerjee},
  title = {{Context-aware Medication Event Extraction from Unstructured Text}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
}
Figure