EMSContExt: EMS Protocol-Driven Concept Extraction for Cognitive Assistance in Emergency Response

Sarah Masud Preum, Sile Shu, Homa Alemzadeh, John A. StankovicAAAI 2020 · DOI

Introduction

Emergency Medical Services (EMS) providers operate in high-stress, high-stakes environments where they must recall and execute specific medical protocols from hundreds of possibilities. While an "intelligent cognitive assistant" could provide real-time guidance by listening to the scene, building such a system is historically difficult due to domain mismatch. Standard medical NLP tools struggle with EMS-specific jargon and shorthand, such as "pt A+Ox4" (patient is alert and oriented) or "skin N/D/W" (normal, dry, and warm). Furthermore, the lack of large-scale, annotated EMS datasets makes automated concept extraction a significant challenge.

Methodology

To address these challenges, researchers developed EMSContExt, a weakly-supervised system designed to curate a domain-specific lexicon without requiring massive manual annotation. The system utilizes a multi-stage pipeline to bridge the gap between general medical knowledge and field-specific communication.

The EMSContExt system architecture, illustrating the integration of knowledge-based and data-driven expansion stages to create an expanded EMS lexicon.

Multi-Stage Expansion

  1. Knowledge-Integrated Expansion: The process begins with "seed" concepts from EMS protocols, expanded using WordNet (lexical), the Unified Medical Language System (medical), and a custom dictionary of 370 unique EMS abbreviations.
  2. Data-Driven Expansion: To capture natural speech patterns, a word2vec model was trained on over 9,400 unstructured EMS narratives. This allows the system to map protocol terms like "trauma" to field-observed synonyms like "bruising" or "abrasions."
  3. Filtering & Ranking: Candidate phrases are refined through stop-word filters, keyword filters, and Part-of-Speech (POS) taggers to eliminate linguistic noise and ensure relevance to the protocol seeds.

Evaluation and Results

EMSContExt was evaluated using a test dataset of 84 cases manually annotated by EMS providers. It was compared against MetaMap, the current state-of-the-art tool for medical concept extraction.

Performance comparison showing EMSContExt significantly outperforming standard medical NLP tools in Recall, Precision, and F1-score.

Key Findings

  • Superior Accuracy: EMSContExt achieved a 0.85 recall and 0.82 F1-score, vastly outperforming MetaMap’s 0.236 F1-score. This demonstrates that domain-specific tuning is more effective than general medical sophistication for field applications.
  • Operational Efficiency: In emergency response, speed is a safety feature. EMSContExt processed cases in approximately 2 seconds, making it 6 times faster than MetaMap (12 seconds). This efficiency allows the system to run on resource-constrained wearable devices in real-time.
  • Protocol Recommendation: When applied to suggesting interventions, the system improved weighted recall by 6% over general-purpose baselines.

Conclusion and Future Work

EMSContExt demonstrates that a combination of expert knowledge and weak supervision can effectively overcome the resource constraints of niche medical domains. By prioritizing speed and domain specificity, the system provides a viable path toward real-time cognitive assistance for first responders.

Future research will focus on improving negation detection (e.g., distinguishing "chest pain" from "no chest pain" in EMS shorthand) and addressing the acoustic noise inherent in chaotic emergency scenes, such as sirens and shouting, to ensure robust performance in the field.

Summary updated

BibTeX
@inproceedings{preum2020emscontext,
  author = {Sarah Masud Preum and Sile Shu and Homa Alemzadeh and John A. Stankovic},
  title = {{EMSContExt: EMS Protocol-Driven Concept Extraction for Cognitive Assistance in Emergency Response}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i08.7048},
}
Figure