Sim911: Towards Effective and Equitable 9-1-1 Dispatcher Training with an LLM-Enabled Simulation

Zirong Chen, Elizabeth Chason, Noah Mladenovski, Erin Wilson, Kristin Mullen, Stephen Martini, Meiyi Ma

This paper presents Sim911, the first Large Language Model (LLM)-powered simulation system specifically designed for training 9-1-1 dispatchers. By automating the role of the "caller," Sim911 provides a scalable, realistic, and equitable training environment that addresses the critical staffing shortages currently plaguing emergency communication centers.

TL;DR

Sim911 is an AI-driven simulation platform that uses LLMs to role-play 9-1-1 callers, providing realistic and diverse training scenarios for dispatchers. It leverages real-world call data and a multi-stage validation loop to ensure simulations are factually accurate and inclusive of underserved populations, significantly reducing the training burden on experienced personnel.

What Problem Does This Paper Address?

Emergency response services in the U.S. are facing a severe staffing crisis, with approximately 25,000 unfilled dispatcher positions nationwide. Traditional training is a major bottleneck: it relies on "role-playing," where an experienced dispatcher must stop their actual emergency duties to act as a caller for a trainee. This process is labor-intensive, inconsistent, and often fails to cover the full spectrum of real-world scenarios.

Furthermore, conventional training often neglects the needs of vulnerable populations—such as non-native English speakers, the unhoused, or individuals experiencing mental health crises. This lack of diversity in training leads to disparities in how emergency services are delivered to different communities. Sim911 aims to bridge this "reality gap" by providing a consistent, high-fidelity, and equitable training tool.

Methods

The authors developed a three-pillar technical framework to transform a standard LLM into a specialized emergency simulator:

  1. Knowledge Construction (KC): The system analyzes thousands of archived 9-1-1 calls to build a structured database. It extracts "Incident Specifications" (e.g., a car crash in severe weather) and "Caller Images" (e.g., an anxious teenager or a non-native speaker).
  2. Context-aware Controlled Generation (CaCG): To prevent the LLM from acting "out of character" or hallucinating, Sim911 uses a combination of:
  3. Retrieval-Augmented Generation (RAG): Pulling real local addresses and protocols from factual databases.
  4. Chain-of-Thought (CoT): Forcing the model to "plan" its responses based on the training objectives.
  5. Few-Shot Prompting (FSP): Providing the model with specific examples of past interactions to mirror realistic speech patterns.
  6. Validation with Looped Correction (VLC): A "co-pilot" system checks the LLM's output in real-time. It verifies formatting, ensures the scenario stays on track, and uses the Google Maps API to confirm that any addresses mentioned actually exist. If a response fails these checks, the system loops back to regenerate it before the trainee ever sees it.

Datasets

The researchers collaborated with the Metro Nashville Department of Emergency Communications (MNDEC) to utilize:

  • 11,841 real-world 9-1-1 call recordings and transcripts (Nov 2022 – May 2024).
  • Computer-Aided Dispatch (CAD) logs for factual grounding.
  • 33 conventional training scripts used as a baseline for human-led training.
  • Validated local address lists and connectivity maps of the Nashville area.

Key Insights and Ideas

  • Grounding is Mandatory: Standard LLMs (like "plain" GPT-4o) are poor at 9-1-1 simulations because they fabricate geographic details. Sim911 proves that strict grounding via RAG and external APIs (like Google Maps) is essential for professional-grade training.
  • Equity as a Core Metric: The authors didn't just measure "accuracy"; they measured "equity." By specifically tagging and retrieving data for vulnerable groups, they ensured the AI could realistically simulate the nuances of high-stress calls from diverse backgrounds.
  • The "Force Multiplier" Effect: In its Nashville deployment, Sim911 saved over 26 hours of active simulation time for senior dispatchers. This allows experienced staff to remain on the floor handling real emergencies while trainees practice with the AI.

Key Results

  • Superior Realism: Sim911 (using GPT-4o) achieved a Perplexity (PPL) of 11.07, significantly better than the baseline of 61.99 for a non-optimized model.
  • High Authenticity: The system achieved a 99.19% success rate in providing factually accurate locations, verified by the Google Maps API.
  • User Approval: In a study with MNDEC personnel, 90% of participants found Sim911 to be as effective as, or even superior to, traditional human-led training.
  • Broad Coverage: The system successfully supports 57 different incident types and 14 specific caller "tags" (e.g., "unhoused," "mental health"), providing a much broader curriculum than typical human-led sessions.

Limitations and Future Work

  • Hallucination Persistence: While the validation loop catches most errors, LLMs can still occasionally generate fabricated information that bypasses automated checks.
  • Knowledge Timeliness: The system relies on static factual bases. Future work will focus on real-time updates to reflect changes in city infrastructure or emergency protocols.
  • Model Exploration: The current system relies heavily on GPT-4o via API. The authors plan to explore fine-tuning open-source models (like Llama or Falcon) to allow for local, private hosting within high-security dispatch centers.

Summary updated

Abstract

Emergency response services are vital for enhancing public safety by safeguarding the environment, property, and human lives. As frontline members of these services, 9-1-1 dispatchers have a direct impact on response times and the overall effectiveness of emergency operations. However, traditional dispatcher training methods, which rely on role-playing by experienced personnel, are labor-intensive, time-consuming, and often neglect the specific needs of underserved communities. To address these challenges, we introduce Sim911, the first training simulation for 9-1-1 dispatchers powered by Large Language Models (LLMs). Sim911 enhances training through three key technical innovations: (1) knowledge construction, which utilizes archived 9-1-1 call data to generate simulations that closely mirror real-world scenarios; (2) context-aware controlled generation, which employs dynamic prompts and vector bases to ensure that LLM behavior aligns with training objectives; and (3) validation with looped correction, which filters out low-quality responses and refines the system performance.
BibTeX
@inproceedings{chen2025sim,
  author = {Zirong Chen and Elizabeth Chason and Noah Mladenovski and Erin Wilson and Kristin Mullen and Stephen Martini and Meiyi Ma},
  title = {{Sim911: Towards Effective and Equitable 9-1-1 Dispatcher Training with an LLM-Enabled Simulation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i27.35006},
}

Loading related content...

Figure