This paper presents Sim911, the first Large Language Model (LLM)-powered simulation system specifically designed for training 9-1-1 dispatchers. By automating the role of the "caller," Sim911 provides a scalable, realistic, and equitable training environment that addresses the critical staffing shortages currently plaguing emergency communication centers.
TL;DR
Sim911 is an AI-driven simulation platform that uses LLMs to role-play 9-1-1 callers, providing realistic and diverse training scenarios for dispatchers. It leverages real-world call data and a multi-stage validation loop to ensure simulations are factually accurate and inclusive of underserved populations, significantly reducing the training burden on experienced personnel.
What Problem Does This Paper Address?
Emergency response services in the U.S. are facing a severe staffing crisis, with approximately 25,000 unfilled dispatcher positions nationwide. Traditional training is a major bottleneck: it relies on "role-playing," where an experienced dispatcher must stop their actual emergency duties to act as a caller for a trainee. This process is labor-intensive, inconsistent, and often fails to cover the full spectrum of real-world scenarios.
Furthermore, conventional training often neglects the needs of vulnerable populations—such as non-native English speakers, the unhoused, or individuals experiencing mental health crises. This lack of diversity in training leads to disparities in how emergency services are delivered to different communities. Sim911 aims to bridge this "reality gap" by providing a consistent, high-fidelity, and equitable training tool.
Methods
The authors developed a three-pillar technical framework to transform a standard LLM into a specialized emergency simulator:
- Knowledge Construction (KC): The system analyzes thousands of archived 9-1-1 calls to build a structured database. It extracts "Incident Specifications" (e.g., a car crash in severe weather) and "Caller Images" (e.g., an anxious teenager or a non-native speaker).
- Context-aware Controlled Generation (CaCG): To prevent the LLM from acting "out of character" or hallucinating, Sim911 uses a combination of:
- Retrieval-Augmented Generation (RAG): Pulling real local addresses and protocols from factual databases.
- Chain-of-Thought (CoT): Forcing the model to "plan" its responses based on the training objectives.
- Few-Shot Prompting (FSP): Providing the model with specific examples of past interactions to mirror realistic speech patterns.
- Validation with Looped Correction (VLC): A "co-pilot" system checks the LLM's output in real-time. It verifies formatting, ensures the scenario stays on track, and uses the Google Maps API to confirm that any addresses mentioned actually exist. If a response fails these checks, the system loops back to regenerate it before the trainee ever sees it.
Datasets
The researchers collaborated with the Metro Nashville Department of Emergency Communications (MNDEC) to utilize:
- 11,841 real-world 9-1-1 call recordings and transcripts (Nov 2022 – May 2024).
- Computer-Aided Dispatch (CAD) logs for factual grounding.
- 33 conventional training scripts used as a baseline for human-led training.
- Validated local address lists and connectivity maps of the Nashville area.
Key Insights and Ideas
- Grounding is Mandatory: Standard LLMs (like "plain" GPT-4o) are poor at 9-1-1 simulations because they fabricate geographic details. Sim911 proves that strict grounding via RAG and external APIs (like Google Maps) is essential for professional-grade training.
- Equity as a Core Metric: The authors didn't just measure "accuracy"; they measured "equity." By specifically tagging and retrieving data for vulnerable groups, they ensured the AI could realistically simulate the nuances of high-stress calls from diverse backgrounds.
- The "Force Multiplier" Effect: In its Nashville deployment, Sim911 saved over 26 hours of active simulation time for senior dispatchers. This allows experienced staff to remain on the floor handling real emergencies while trainees practice with the AI.
Key Results
- Superior Realism: Sim911 (using GPT-4o) achieved a Perplexity (PPL) of 11.07, significantly better than the baseline of 61.99 for a non-optimized model.
- High Authenticity: The system achieved a 99.19% success rate in providing factually accurate locations, verified by the Google Maps API.
- User Approval: In a study with MNDEC personnel, 90% of participants found Sim911 to be as effective as, or even superior to, traditional human-led training.
- Broad Coverage: The system successfully supports 57 different incident types and 14 specific caller "tags" (e.g., "unhoused," "mental health"), providing a much broader curriculum than typical human-led sessions.
Limitations and Future Work
- Hallucination Persistence: While the validation loop catches most errors, LLMs can still occasionally generate fabricated information that bypasses automated checks.
- Knowledge Timeliness: The system relies on static factual bases. Future work will focus on real-time updates to reflect changes in city infrastructure or emergency protocols.
- Model Exploration: The current system relies heavily on GPT-4o via API. The authors plan to explore fine-tuning open-source models (like Llama or Falcon) to allow for local, private hosting within high-security dispatch centers.
Summary updated