Abstract
As the COVID-19 pandemic spreads across the world, it is important to understand its features and responses to public health interventions in real-time. The field of infectious diseases epidemiology has highly advanced modeling strategies that yield relevant estimates. These include the doubling time of the epidemic and various other representations of the numbers of cases identified over time. Crude estimates of these quantities suffer from dependence on the underlying testing strategies within communities. We clarify the functional relationship between testing and the epidemic parameters, and thereby derive sensitivity analyses that explore the range of possible truths under various testing dynamics. We derive the required adjustment to the estimates of interest for New York City. We demonstrate that crude estimates that assume stable testing or complete testing can be biased.
Introduction
In the early stages of a viral outbreak, the "epidemic curve"—a plot of confirmed cases over time—serves as the primary instrument for public health decision-making. However, this curve is often a "foggy lens." Because confirmed cases are a function of both the virus's spread and the availability of diagnostic tests, fluctuations in testing capacity can create a misleading picture of the epidemic's trajectory. If testing fails to keep pace with the virus, the curve may appear to flatten even as the actual infection rate accelerates. This paper addresses this fundamental flaw by quantifying the bias in crude epidemic metrics and providing a framework for adjustment.
The Problem of Crude Metrics
A central metric in epidemiology is the doubling time ($T_d$), which represents the time required for the total number of cases to double. While $T_d$ is a more intuitive measure of growth than raw case counts, it remains inextricably linked to testing dynamics. If the probability of an infected individual being tested decreases over time, the observed doubling time will appear longer (more optimistic) than the true doubling time. Without accounting for these shifts, policy decisions regarding social distancing and resource allocation may be based on a false sense of security.
Methodological Framework
The researchers propose a non-parametric approach rooted in Bayes' Theorem to relate the observed number of cases ($O(t)$) to the true, unknown number of infections ($C(t)$). The framework introduces several key components:
1. The Testing Ratio ($R$)
The authors define a ratio, $R(t, d)$, representing the change in the probability of an infected person being tested between two time points.
- If $R = 1$, testing is stable.
- If $R > 1$, testing probability has decreased, leading to an overestimation of doubling time.
- If $R < 1$, testing probability has increased, leading to an underestimation of growth.
2. Deaths as a Proxy for Infections
Assuming the Infection Fatality Rate (IFR) remains relatively constant over short periods, the ratio of deaths to true cases should remain stable. By comparing the ratio of deaths to observed cases at different intervals, the authors can estimate the testing ratio $R$. This requires incorporating a lag factor ($l$)—estimated at 18 days for COVID-19—to account for the duration between a positive test (or symptom onset) and death.
3. Sensitivity Analysis
In regions where death data is unavailable or unreliable, the paper suggests a sensitivity analysis. Researchers can input hypothetical decreases in testing (e.g., 10%, 20%, 25%) to determine how much the doubling time estimate would shift under various scenarios, providing a range of possible truths rather than a single biased point estimate.
Data and Results
The study analyzed data from the early 2020 period using the COVID Tracking Project (covering 12 US states with high caseloads) and the NYC Health Department.
The Magnitude of Bias
The analysis revealed that failing to account for even a 25% drop in testing probability could lead to an overestimation of doubling time by as much as three days. In the context of an exponential pandemic, a three-day discrepancy in growth rate is significant for hospital capacity planning and policy timing.
The New York City Case Study
Interestingly, New York City provided a counter-example to the common trend of decreasing testing probability. During late March and early April 2020, the data suggested that the probability of testing infected individuals actually increased slightly. Consequently, crude estimates for NYC were overly pessimistic; the epidemic was slowing down slightly faster than the raw case numbers suggested. This highlights that the "testing trap" can bias results in either direction.
Limitations
While the proposed framework offers a more robust way to track epidemics, it faces several constraints:
- Retrospective Nature: The 18-day lag required for death data means this method primarily provides a retrospective correction rather than a real-time adjustment for today's numbers.
- Data Quality: The accuracy of the adjustment depends on the reporting of "probable" deaths. If a jurisdiction only records deaths confirmed by a lab test, the death data itself becomes subject to the same testing bias the model seeks to correct.
- Healthcare Constraints: The assumption of constant lethality may fail if a healthcare system becomes overwhelmed, causing the death rate to rise independently of the infection rate.
Conclusion
Estimating epidemic parameters without accounting for testing dynamics is inherently risky. By utilizing death counts as a proxy and performing sensitivity analyses, researchers can uncover the true doubling time of an outbreak. This paper demonstrates that stable testing is a rare luxury in a burgeoning pandemic, and mathematical adjustments are essential for an accurate understanding of public health interventions.
Summary updated