Real-time flu tracking blends several data streams: doctor-visit reports, emergency-department complaints, and web-search volume. Most of these streams are only available for large areas such as a county or a state. A model that converts them into neighborhood-level estimates works well in wealthy neighborhoods and badly in poor ones. Socioeconomic bias in influenza surveillance showed this across Dallas–Fort Worth zip codes. Their regularized regression models predicted influenza hospitalizations best in the most affluent poverty quartile and worst in the poorest, and the difference was significant (P < 0.0001). Adding Google Flu Trends and syndromic emergency-department data did not close the gap.
Their most interesting finding rules out the easy explanation. Hospitalizations in poor zip codes were more synchronized with each other than in affluent ones, so the failure is not erratic local epidemics. It is a measurement problem. The regional streams mostly record the people who search online and see primary-care doctors, and those people are disproportionately affluent.
We propose to model that measurement problem directly instead of regressing around it. Each regional stream is treated as a demographically weighted sum of neighborhood epidemics. The weights are described by a handful of census covariates per stream, not learned separately for each zip code. Because the weights are shared and low-dimensional, the model can be fit with only a few seasons of data. Inverting them lets the model separate the part of a regional signal that speaks for poor neighborhoods from the part that mostly echoes affluent ones.
Core Claim
The poverty gap in multi-source nowcasting arises because current methods fit a direct mapping from regional streams to local burden. Examples are ARGO, a sparse regression on search terms plus seasonal autoregression, and the Poisson additive models of Scarpino et al. A direct mapping implicitly assumes every neighborhood is represented in each stream in proportion to its population. When that assumption is wrong, the mapping learned where data are rich is transferred wrongly to where they are poor.
We claim that an explicit observation model will shrink the error gap between the poorest and richest quartiles without hurting the affluent quartile. In this model, each stream's coverage of a neighborhood (what fraction of its infections the stream "sees") and its delay are smooth functions of that neighborhood's demographics. The model is fit under the small-sample conditions of a handful of seasons. It also produces an interpretable, falsifiable by-product: estimated coverage curves showing, for example, that search coverage falls with poverty faster than emergency-department coverage does.
The core mechanism fits in one equation. Let $\lambda_{z,t}$ be the latent influenza incidence in zip code $z$ during week $t$. Let $x_z$ be a short vector of that zip code's census covariates: poverty rate, uninsured share, age-65+ share, and household broadband share. The regional value of stream $s$ in week $t$ is $y_{s,t}$, and it is modeled as
$$y_{s,t} = \sum_{z} N_z \, w_s(x_z)\, \lambda_{z,\,t-\ell_s(x_z)} + \varepsilon_{s,t}, \qquad \log w_s(x_z) = \alpha_s + \beta_s^{\top} x_z .$$
Here $N_z$ is the population of zip code $z$. The coverage $w_s$ and the delay $\ell_s$ (in weeks) depend on demographics through a few parameters per stream. The noise term $\varepsilon_{s,t}$ is the stream's measurement error.
Zip-level hospitalizations $h_{z,t}$ serve as a high-coverage anchor, modeled as $h_{z,t} \sim \text{Poisson}(\rho_z \lambda_{z,t})$, where $\rho_z$ is the zip code's hospitalization ratio. The latent incidence is written as a regional epidemic curve scaled per zip code, plus a low-rank deviation. That structure is the one Scarpino et al.'s synchrony finding supports.
The total parameter count grows with the number of streams and covariates, not with the number of zip codes. This is what makes a few seasons of data enough.
Research Program
The work has three parts.
Build the estimator. The model is fit with a state-space approach, estimating the latent curve week by week and the coverage parameters by maximum marginal likelihood over a rolling training window. Nowcasts for zip code $z$ come from the posterior of $\lambda_{z,t}$, mapped through the hospitalization ratio $\rho_z$, so the model reports intervals rather than single numbers. Delay parameters are constrained to a small grid of zero to three weeks to keep identification honest.
Establish identifiability before trusting any coverage estimate. If coverage weights and local incidence could trade off freely, the model would not be identified. The hospitalization anchor and the shared-curve structure are what pin them down. We will run a simulation study first. It plants known coverage gradients into synthetic zip-level epidemics with the observed synchrony structure, then measures how well the slopes $\beta_s$ are recovered as the number of seasons shrinks from five to two. If recovery fails at realistic season counts, we will report that as a negative result rather than proceed.
Test whether the inversion survives real-time data. Regional streams are revised after their first release. Delphi-RF showed that these revisions are non-stationary and that they matter most in the first few days after release. We will feed the model the first, unrevised releases where they can be reconstructed. ARGO's robustness to initial CDC releases suggests some streams tolerate this well, but nothing in the file speaks to whether revision behavior differs by population.
The role of each cited line of work should be clear. Scarpino et al. motivates the work: it locates the gap and rules out the asynchrony explanation. ARGO and ARGONet supply the strongest competitors. ARGONet adds an athenahealth electronic-health-record stream and a network model that borrows from neighboring states, choosing between the two on a rolling 104-week window. Both learn direct mappings and treat coverage as uniform.
Transfer Survival Boosting supplies the transfer idea we compete against. It re-weights large source cohorts toward a small target cohort to avoid negative transfer, but it handles static survival data, not regional time series. MiME supplies a precedent for the design principle. It hard-codes a known data-generating hierarchy (visit, diagnosis, treatment) into an electronic-health-record model and gains most on the smallest data subsets. Here the known structure is "who is counted by which stream," not a coding hierarchy.
Correcting Ascertainment Bias in Passive Disease Surveillance, Cameron et al., and the Beijing incidence study constrain the design. They correct under-ascertainment with fixed, retrospective multipliers. Our coverage terms are the dynamic, covariate-indexed analogue of those multipliers, and the static versions become a baseline.
Baselines
Naive reference points. The first is a seasonal autoregression on each zip code's own lagged hospitalizations, which uses no other streams. The second is a historical week-of-season average. ARGONet's state-level comparison already shows that such autoregression is beatable: ARGONet beat the 52-week autoregressive baseline in 36 of 37 states. Whether that holds at zip level in poor neighborhoods is exactly what we test, since a stream that does not cover the poor may add nothing there.
The original model in the gap paper. This is Scarpino et al.'s Poisson additive model with L1-regularized splines over ILINet, BioSense 2.0, and Google Flu Trends. It is the direct evidence of the gap. The file reports the gap's significance (P < 0.0001) and the poverty-quartile boundaries but no quartile-specific error values. We will therefore reproduce it on identical splits and treat our reproduction as the reference.
The strongest existing multi-source method. This is an ARGONet-style ensemble adapted to zip codes, fit per zip code on the same streams. The first component is an ARGO-type sparse regression with autoregressive terms on a two-year rolling window. The second is a network component that borrows from the most correlated zip codes. A rolling winner-takes-all selector picks between them. Published state-level results for this family are the 36-of-37 win over autoregression and a reduction of more than 50% in root-mean-squared error versus Google Flu Trends in 23 states, both from ARGONet. No reported result was found on Lacuna for ARGONet at zip-code level or stratified by poverty.
Two transfer baselines. One is a TSB-inspired variant that re-weights affluent-zip training weeks by their similarity to the target zip's covariate profile. The other is a hierarchical model that shrinks per-zip regression coefficients toward a common mean, with no demographic information.
Fairness of comparison. All methods see the same streams, the same training windows, and the same rolling-origin splits. Each gets the same hyperparameter search budget, tuned on a held-out season and never on the test season.
What counts as beating them. We count a win if three things hold together. First, the population-normalized error in the poorest quartile is lower than the best competitor's. Second, the gap between the poorest and richest quartiles narrows, judged by Scarpino et al.'s permutation test. Third, affluent-quartile error does not worsen beyond a tolerance fixed before analysis.
Ablation. The key ablation replaces the covariate-dependent coverage $w_s(x_z)$ with one constant per stream, keeping everything else identical. If that variant matches the full model, then demographic weighting of the streams is not doing the work, and our claim is wrong.
Why not just…?
Why not just pool the poor zip codes together, since they are synchronized? This is the most serious threat. Scarpino et al.'s synchrony result suggests that fitting one model to the whole poorest quartile could recover much of the lost accuracy simply by adding sample size. We will run quartile-pooled versions of both the original additive model and the ARGO-style regression as first-class baselines.
Our model contains quartile pooling as a special case: coverage that is a step function of poverty quartile. If the pooled baseline matches us, the honest conclusion is that the gap is mainly a sample-size problem, not a coverage problem. The contribution would then shrink to the coverage-curve diagnostics. We expect to win where quartile pooling is coarse: zip codes near quartile boundaries, covariates other than poverty (broadband, uninsured share), and weeks when one stream is missing. We will report results separately for these cases.
Why not just add poverty-by-stream interaction terms to the existing regression? That lets coverage vary with poverty, but it stays a direct mapping fit separately for each zip code. Each zip code must then estimate many interaction coefficients from a few seasons, which is the small-sample failure that MiME and TSB both address by adding structure or transfer. It also cannot express the identity that a regional stream is a sum over its constituent neighborhoods. We include this variant, tuned under the same budget. If it ties our model, the observation-model structure is unnecessary.
Why not just transfer from affluent zip codes with instance re-weighting or fine-tuning? Transfer moves the mapping learned in affluent zip codes, and that mapping is what encodes affluent coverage. Transfer Survival Boosting itself names negative transfer under divergent feature relationships as a limitation. Portable clinical representations report that learned representations degrade across sites. The re-weighting baseline tests this directly. We predict it will help the middle quartiles and fail the poorest, where no affluent weeks resemble the target.
Why not just apply static under-reporting multipliers? Fixed ratios of the kind used for Zika, chlamydia, and Beijing influenza assume constant care-seeking behavior and constant delays. Delphi-RF's evidence of non-stationary revision patterns argues against that assumption. A per-quartile constant multiplier on the regional prediction is included as a baseline. It is essentially our ablation without the delay terms or the hospitalization anchor.
Proposed Experiments
The primary benchmark reconstructs the Scarpino et al. setting. It covers six Dallas–Fort Worth counties over 2007–2012, uses THCIC zip-level influenza discharges as the target, and uses ILINet, BioSense 2.0, and Google Flu Trends as regional inputs. Poverty quartiles come from the 2010 Census and the 2011 American Community Survey. Access to THCIC records is the main practical risk.
Evaluation is rolling-origin, leaving one season out and including the 2009 pandemic as a deliberate stress test of an atypical season. We measure population-normalized out-of-sample root-mean-squared error per quartile, the gap between the poorest and richest quartiles with a permutation test, and the coverage of 80% prediction intervals per quartile. EpiFNP showed that calibration, not just accuracy, separates methods in atypical seasons, which is why interval coverage is included.
A secondary benchmark replicates the structure at state level. It uses ILINet, athenahealth, and Google search data across the ARGONet states, grouped by state poverty and broadband share, and asks whether the coverage effect appears at coarser resolution where ARGONet is the published standard.
Three checks are decisive.
Coverage pattern. If the coverage interpretation is right, the estimated coverage slope for Google Flu Trends should be clearly more negative in poverty than the slope for BioSense. The model should also gain most in the weeks when search data carry the most weight. If the slopes come out flat or reversed and accuracy still improves, the gain comes from something other than coverage, and we would say so.
Ablation. The constant-coverage ablation must lose to the full model in the poorest quartile.
Quartile pooling. The quartile-pooled baseline must lose at quartile boundaries and in weeks with missing streams. If it does not, the paper becomes a careful negative result showing that pooling within strata, not demographic deconvolution, is what fixes the surveillance gap. That would still be worth knowing.
References
- Socioeconomic bias in influenza surveillance
- Accurate estimation of influenza epidemics using Google search data via ARGO
- Real-time forecasting of data revisions in epidemic surveillance streams
- Improved state-level influenza nowcasting in the United States leveraging Internet-based data and network approaches
- Boosting Transfer Learning with Survival Data from Heterogeneous Domains
- MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare
- On the origin and timing of Zika virus introduction in Brazil
- Estimating incidence of sexually transmissible infections in Australia
- Estimated incidence and number of outpatient visits for seasonal influenza in 2015–2016 in Beijing, China
- Can we generate portable representations for clinical time series data using LLMs?
- When in Doubt: Neural Non-Parametric Uncertainty Quantification for Epidemic Forecasting