An AI-Based Public Health Data Monitoring System

Ananya Joshi, Nolan Gormley, Richa Gadgil, Tina Townes, Roni Rosenfeld, Bryan WilderarXiv 2025 · DOI · arXiv

Abstract

Traditional public health monitoring relies on rigid, threshold-based alerts that often lead to alert fatigue and missed signals in high-volume data streams. This paper introduces an AI-driven ranking system developed by researchers at Carnegie Mellon University to overhaul national health data monitoring. By shifting from binary alerts to a continuous anomaly ranking paradigm, the system provides reviewers with the necessary situational awareness to distinguish reporting errors from genuine outbreaks. Deployed to monitor up to 5 million daily data points, the system achieved a 54x increase in reviewer efficiency and a 5x improvement over basic AI tools, demonstrating the critical importance of human-centered interface design in algorithmic monitoring.

Introduction

The "data modernization" of public health has created a paradox: while more data is available than ever before, the tools to monitor it remain stuck in the past. Current systems typically utilize threshold-based alerting, where notifications are triggered only when specific statistical limits are exceeded. This approach suffers from three primary flaws: stagnation due to the need for constant manual tuning, a lack of context for individual alerts, and severe alert fatigue. When reviewers are faced with tens of thousands of alerts weekly, they often revert to manual searching, which is inefficient and prone to error.

Methods

The proposed system replaces binary alerts with a ranking-based monitoring paradigm. Using an unsupervised AI approach, every data point is assigned a continuous anomaly score based on its deviation from expected patterns. This allows reviewers to focus on the most significant events first.

Human-Centered Design

The system was developed through an 18-month collaborative process involving methodologists, engineers, and data reviewers to ensure the tool met the practical needs of public health officials.

The collaborative design process involving methodologists, engineers, and data reviewers.

Interface Innovations

The interface incorporates several key modules to enhance situational awareness:

  • Filtering (M1): Enables segmentation by region or provider.
  • Situational Awareness Panels (M2): Utilizes choropleth hex maps and aggregated scores to compare local anomalies with regional or national trends.
  • Data Evolution (M3): Employs 1D heatmaps to track how scores change as data is revised over time.
The monitoring dashboard featuring a US hex map for situational awareness and detailed indicator charts.

Datasets and Evaluation

The system was evaluated using real-world, national-level public health data curated by the Delphi Research Group at CMU. This includes monitoring up to 5,000,000 data points daily across indicators such as COVID-19 cases, hospitalizations, deaths, and syndromic surveillance data. A three-month longitudinal study compared the new system against two baselines: a manual exploratory tool and a basic AI ranking tool without the advanced interface.

Results and Discussion

The longitudinal evaluation yielded impressive quantitative gains. Reviewers were 54 times faster at recording events compared to the traditional manual exploratory method. Even compared to a simple AI ranking list, the full system was 5 times faster, proving that the interface is just as critical as the algorithm.

Key insights from the deployment include the rejection of the "Prescriptive Alerting Fallacy"—the idea that anomalies can be perfectly defined beforehand. Instead, ranking allows the AI to surface "unknown unknowns" for human experts to contextualize. Furthermore, the system enabled the identification of "meta-events," or complex patterns across multiple regions or indicators that were previously impossible to spot.

Conclusion

By prioritizing human-centered design and situational awareness, this AI-driven system significantly improves the efficiency and efficacy of public health monitoring. While successful, future work will focus on expanding data dimensionality (e.g., age subgroups), incorporating automated narrative summarization via Large Language Models, and improving sequence detection across different health indicators.

Summary updated

Abstract

Public health experts need scalable approaches to monitor large volumes of health data (e.g., cases, hospitalizations, deaths) for outbreaks or data quality issues. Traditional alert-based monitoring systems struggle with modern public health data monitoring systems for several reasons, including that alerting thresholds need to be constantly reset and the data volumes may cause application lag. Instead, we propose a ranking-based monitoring paradigm that leverages new AI anomaly detection methods. Through a multi-year interdisciplinary collaboration, the resulting system has been deployed at a national organization to monitor up to 5,000,000 data points daily. A three-month longitudinal deployed evaluation revealed a significant improvement in monitoring objectives, with a 54x increase in reviewer speed efficiency compared to traditional alert-based methods. This work highlights the potential of human-centered AI to transform public health decision-making.
BibTeX
@misc{joshi2025ai,
  author = {Ananya Joshi and Nolan Gormley and Richa Gadgil and Tina Townes and Roni Rosenfeld and Bryan Wilder},
  title = {{An AI-Based Public Health Data Monitoring System}},
  year = {2025},
  eprint = {2506.04429},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.04429},
}
Figure