Introduction
Emergency Communication Centers (DECs) are currently facing unprecedented operational pressure. While 911 services are reserved for life-threatening emergencies, 311 systems handle a high volume of municipal non-emergency issues, such as road damage or stolen property. Often, the same personnel manage both streams, meaning a surge in 311 calls can directly delay responses to critical 911 emergencies.
Existing automated dialogue systems, such as AWS Lex or Google DialogFlow, typically rely on rigid, pre-defined flowcharts that struggle with the fluid nature of real-world calls. Research indicates that 72% of callers provide "unintentional additional information" early in a conversation, and incident types shift or multiply in 38% of cases. Auto311 is designed to address these complexities using a multi-modular, confidence-guided architecture that dynamically extracts information and predicts incident types from messy, real-world dialogues.
System Architecture
Auto311 employs a modular design to ensure both efficiency and safety. The system begins with a conversational interface utilizing OpenAI’s Whisper for speech-to-text and Suno-AI Bark for text-to-speech.
As shown in the architecture above, the system includes an "always-on" handover control layer. This safety mechanism uses Latent Dirichlet Allocation (LDA) and NLP rules to detect high-urgency keywords or caller frustration, immediately routing the call to a human operator if necessary. The core logic resides in the interaction between Information Itemization and Incident Type Prediction, governed by confidence thresholds ($λ_1, λ_2$).
Confidence-Guided Report Generation
The primary innovation of Auto311 is its use of confidence scores to navigate the dialogue. Instead of a simple classifier, the Incident Type Prediction module uses a multi-layer hierarchical structure with Monte Carlo Dropout to measure consistency across trials.
If the system identifies a field or incident type with low confidence ($conf \leq λ$), it proactively asks for clarification. Conversely, if confidence is high, the system strategically skips irrelevant questions to streamline the call. This approach allows the system to "harvest" information from long-winded explanations, filling out future parts of the report before the specific question is even asked. As illustrated in the figure above, the system can dynamically update, merge, or even delete incident types (e.g., shifting from "Damaged Property" to "Lost Stolen") as more context is gathered.
Evaluation and Key Results
The researchers evaluated Auto311 using a unique dataset of 11,796 authentic non-emergency call recordings from the Nashville Department of Emergency Communications. For information itemization, the model was benchmarked against standard datasets like SQuAD and CUAD.
Key performance metrics include:
- Accuracy: Achieved an average F1-score of 92.54% for incident prediction across categories like noise violations and abandoned vehicles.
- Consistency: Reached a 0.93 average consistency score in information extraction compared to ground-truth human reports.
- Efficiency: Emulations demonstrated that the system successfully decreased the number of conversation turns as the length of the caller's initial utterance increased, proving its ability to process proactive information.
- Superiority over General LLMs: While GPT-3.5 and GPT-4 are powerful, Auto311—fine-tuned on domain-specific data—outperformed them in information itemization by up to 47%.
Conclusion and Future Work
Auto311 demonstrates the potential for task-specific, confidence-guided systems to alleviate the burden on emergency dispatchers. However, several challenges remain. Future work will focus on integrating GIS for better landmark-to-address translation and developing redundancy checks to merge multiple reports regarding the same incident. Currently, the system has been tested via emulations; live, interactive trials with real callers represent the next critical step in validating its real-world utility.
Summary updated