Syndromic surveillance: STL for modeling, visualizing, and monitoring disease counts

Ryan P. Hafen, David E. Anderson, William S. Cleveland, Ross Maciejewski, David S. Ebert, Ahmad M. Abusalah, Mohamed Yakout, Mourad Ouzzani, Shaun J. GrannisBMC Medical Informatics Decis. Mak 2009 · DOI

Abstract

The STL decomposition method for chief complaint counts leads to a rapid and accurate detection method for disease outbreaks, and requires only 90 days of historical data to be put into operation. The visualization tools that accompany the decomposition and outbreak methods provide much insight into patterns in the data, which is useful for surveillance operations.

Introduction

Public health officials rely on "syndromic surveillance"—monitoring pre-diagnostic data like Emergency Department (ED) "chief complaints"—to spot outbreaks, such as influenza or bioterrorism events, before official diagnoses are confirmed. However, this data is inherently noisy. Visit counts fluctuate based on the day of the week, the season, and long-term population trends. Traditional statistical models like EARS or ARIMA often require years of historical data or oversimplify these patterns, leading to low sensitivity or excessive false alarms. The challenge lies in accurately modeling systematic variations so that genuine "spikes" can be identified as potential outbreaks.

Methodology: STL Decomposition

The authors propose using Seasonal-Trend decomposition based on Loess (STL), a non-parametric technique that breaks down daily count data into four distinct parts:

  1. Inter-annual ($T_t$): Long-term trends, such as population growth.
  2. Yearly-seasonal ($S_t$): Annual cycles like the influenza season.
  3. Day-of-the-week ($D_t$): Recurring weekly patterns.
  4. Random-error/Noise ($N_t$): The residual variation after accounting for systematic components.
Figure 3: STL decomposition of respiratory chief complaint counts into inter-annual, yearly-seasonal, day-of-the-week, and noise components.

A key innovation is the application of a square root transformation to raw counts, which stabilizes variance and makes the noise component nearly normal. Furthermore, the use of Local Regression (Loess) provides flexibility; it uses a moving window to fit curves, allowing the model to adapt to changing patterns (like bimodal flu seasons) without needing a decade of history.

Data and Evaluation

The study utilized real-world data from the Indiana Public Health Emergency Surveillance System (PHESS), covering 76 Emergency Departments from 2004 to 2008. To evaluate the model, the authors "injected" simulated outbreaks into the baseline data using a Sartwell lognormal model, mimicking the typical rise and fall of an epidemic curve.

Key Results and Visual Insights

The STL method demonstrated superior performance across several metrics:

  • High Sensitivity: In simulations, STL consistently achieved the highest sensitivity, outperforming EARS and GLM by roughly 10% at low outbreak magnitudes.
  • Rapid Deployment: Unlike sophisticated models requiring 3-8 years of data, STL performs effectively with just 90 days of history.
  • Visual Utility: The decomposition allows officials to see "pure" signals. For instance, the "Monday peak" effect is clearly visible across different departments, helping distinguish routine busy days from genuine threats.
Figure 6: Day-of-the-week components across multiple emergency departments, highlighting the consistent peak on Mondays (Day 1).

The authors also noted that STL accurately tracks complex, bimodal yearly seasonal shapes that rigid parametric models often miss.

Limitations and Future Work

The current approach works best when counts are relatively high. For very rare events (counts near zero), the square root transformation and normality assumptions may not hold. Future research could explore replacing the square root approach with local Poisson maximum likelihood fitting for low-count scenarios and testing the model against different outbreak "shapes."

Summary updated

BibTeX
@article{hafen2009syndromic,
  author = {Ryan P. Hafen and David E. Anderson and William S. Cleveland and Ross Maciejewski and David S. Ebert and Ahmad M. Abusalah and Mohamed Yakout and Mourad Ouzzani and Shaun J. Grannis},
  title = {{Syndromic surveillance: STL for modeling, visualizing, and monitoring disease counts}},
  journal = {BMC Medical Informatics Decis. Mak.},
  year = {2009},
  doi = {10.1186/1472-6947-9-21},
}
Figure