TL;DR
Epira is an automated interviewing system that combines the natural language capabilities of Large Language Models (LLMs) with a deterministic, rule-based controller to conduct public health contact tracing. By enforcing strict protocols, it achieves significantly higher data completeness than open-ended chatbots while maintaining high user trust and usability scores.
What Problem Does This Paper Address?
Case investigation and contact tracing are the backbones of infectious disease control, yet they are notoriously labor-intensive and difficult to scale during outbreaks. Existing digital solutions fall into two "uncanny valleys":
- Web Forms: These are structured but rigid. They fail when respondents need clarification or are hesitant to disclose sensitive information.
- Open-ended Chatbots: While conversational, they are unreliable for clinical use. They often skip mandatory questions, provide unsafe advice, or wander away from established medical protocols.
The authors sought to build a system that offers the "human-like" flexibility of a chatbot with the "protocol-perfect" reliability of a structured form.
Methods
The core innovation of Epira is its backend system architecture, which decouples the "brain" (the LLM) from the "rules" (the controller).
- The Controller: A deterministic Finite State Machine (FSM) manages the interview stages (e.g., identity, symptoms, contacts). It assembles prompts for the LLM, validates the LLM's proposed updates to the case record, and enforces "deterministic rails" to ensure no required fields are skipped.
- The LLM: The system was tested with both a local open-source model (Qwen2.5) and a cloud-based model (GPT-5.4). The LLM's job is to draft natural language replies and extract structured data from the user's responses.
- Adaptability: Epira is disease-agnostic. Administrators can upload a new questionnaire, and the system automatically generates a structured interview specification.
- Human-in-the-loop: The UI features a live case record that the user can edit. If the system detects an emergency or a request for a human, it triggers an immediate handoff to public health staff.
Datasets
The researchers evaluated Epira through two distinct studies:
- Study 1 (Simulated): 108 unique personas (36 routine and 72 "challenging" cases) were created for measles and gonorrhea. These personas were interviewed by Epira (using both Qwen and GPT backends) and by open-ended chatbot baselines, totaling 864 sessions.
- Study 2 (Human Pilot): 72 human volunteers from Prolific role-played fictional cases to assess the system's usability, trust, and completion rates in a 2x2 design (disease x model).
Key Insights and Ideas
- Protocol > Model Choice: One of the most significant findings was that the controller, not the specific LLM, drove performance. Even the smaller, local Qwen2.5 model performed admirably when constrained by Epira’s rules, often matching or nearing the performance of the much larger cloud model.
- The Failure of Open-Endedness: Without protocol control, LLMs are poor interviewers. Open-ended versions of the same models captured 31–42% fewer required fields and ended interviews prematurely.
- Privacy-Preserving Utility: Because the system works well with local models, health departments can deploy Epira on their own hardware. This mitigates the privacy risks associated with sending sensitive health data to third-party cloud providers.
- Conversational Preference: Users overwhelmingly preferred the automated interview over filling out a long online form, suggesting that the "conversational" aspect reduces the cognitive burden of reporting.
Key Results
Epira demonstrated high technical efficacy and user acceptance:
- Data Completeness: In simulated trials, Epira with a local LLM captured 88.9% of required fields and 60.1% of ground-truth contacts.
- Efficiency: Epira was significantly more thorough than open-ended baselines, which struggled to recover even half of the necessary data.
- Usability: Human volunteers gave Epira a mean System Usability Scale (SUS) score of 83/100, which is considered "excellent."
- Trust: The primary trust index was 3.85/5, with users reporting they felt "understood" by the tool.
Limitations and Future Work
While promising, the authors note several limitations:
- Simulated Resistance: The "challenging" personas were simulated; real-world resistance from actual patients may be higher or more nuanced.
- Demographics: The pilot was limited to English-speaking users in the US. Future work is needed to evaluate the system in different languages and across diverse demographic groups.
- Voice Channel: Although the architecture supports voice, the studies focused primarily on the text interface. Evaluating the "voice-to-voice" experience is a key next step for accessibility.
Summary updated