When a disease spreads unevenly, a few infected people cause most new cases. This shows up in daily case counts as extra variance: counts jump around more than they would if everyone transmitted at the same rate. Several methods now estimate this unevenness directly from aggregate case counts. The catch is that bad reporting also makes counts jump around. Weekend pauses, backlogs dumped on a Monday, and irregular reporting intervals all add variance. One of these methods says so explicitly: reporting irregularities inflate its heterogeneity index (Assessing the heterogeneity in the transmission of infectious diseases from time series of epidemiological data). So today's real-time superspreading monitors can confuse a messy reporting system with an outbreak driven by superspreaders.
This note proposes a model that separates the two sources of variance using a structural difference between them. Variance from transmission persists. An unusually large cluster of infections today produces more infections one generation later, so the excess propagates forward through the epidemic's own renewal dynamics. Variance from reporting is transient and conserves mass. A backlog dump moves cases that already existed from one reporting window to another, without creating cases or changing future transmission. We build a continuous-time latent transmission model with a time-varying dispersion parameter $k_t$. Each irregular report enters the likelihood as a count over its own time window, passed through an explicit delay and batching process. The test is whether this recovers $k_t$ under realistic reporting corruption where existing estimators are biased. The test runs on simulations where the true $k_t$ is known and on historical datasets where the reporting schedule can be re-coarsened.
Background and the gap
The standard description of uneven transmission is the negative binomial offspring distribution. Each case's number of secondary infections has mean $R$, and a dispersion $k$ in which small values mean most cases infect nobody while a few infect many (Grenfell et al.; Pooled testing of traced contacts under superspreading dynamics). Estimating $k$ traditionally needs line lists from contact tracing, and these break down during explosive outbreaks.
Two methods supply the starting point by estimating heterogeneity from incidence alone. The Instant-Individual Heterogeneity (IIH) model, from Monitoring real-time transmission heterogeneity from incidence data, extends the discrete renewal equation by making each individual's reproduction number Gamma-distributed. It fits $R_t$ and $k_t$ jointly in sliding windows and is robust to uniform underreporting. However, it assumes a clean daily grid, and its robustness test covered uniform thinning of cases, not the structured batching that real systems produce. The Effective Aggregate Dispersion Index (EffDI) infers the minimal dispersion needed to explain observed fluctuations. Its authors acknowledge the reporting-artifact problem but do not model it. Both papers motivate this proposal. EffDI also constrains it: any estimator that reads dispersion off variance alone will fail in the same way.
On the machine learning side, continuous-time sequence models handle irregular timestamps. These include Neural ODEs, which learn a vector field and integrate it between arbitrary time points (Neural Ordinary Differential Equations). Neural CDEs drive a latent state with a spline built from the observations (Neural Controlled Differential Equations for Irregular Time Series). Jump ODEs evolve smoothly between observations and update discretely when data arrive (Neural Jump Ordinary Differential Equations). The closest epidemic application is EARTH (EARTH). It embeds SIR dynamics in a Neural ODE driven by NCDE paths and stays accurate with 40% of points missing. EARTH targets mean forecasts with homogeneous compartmental dynamics. Its spline inputs also smooth away exactly the spikes that carry the dispersion signal. GN-ODE (Neural Ordinary Differential Equations for Modeling Epidemic Spreading) assumes complete, regular simulated cascades. The Ct-Transformer (Estimating the time-varying effective reproduction number via Cycle Threshold-based Transformer) uses viral-load data but estimates only $R_t$ on a regular grid. None of these models separates reporting variance from transmission variance, and none estimates $k_t$.
Core Claim
Reporting artifacts and superspreading can be separated from aggregate counts alone, without line lists, because they leave different signatures.
Transmission excess propagates. A surplus of infections at time $s$ raises expected incidence at $s + \tau$ in proportion to the generation-interval density $w(\tau)$. Reporting excess conserves mass over the delay window, so it creates negative correlation between adjacent reporting windows and no forward propagation.
A model that encodes both mechanisms explicitly should estimate $k_t$ without bias under reporting corruption that inflates heterogeneity in IIH and EffDI. A homogeneous continuous-time model with a flexible noise head cannot achieve this, because it has no mechanism that ties excess variance to future incidence.
The model
Latent infections $I(s)$ live on a fine time grid, with half-day steps. Expected new infections at time $t$ come from a Gamma-mixed renewal process, following IIH:
$$\lambda(t) = \sum_{s<t} R_s\, \nu_s\, w(t-s), \qquad \nu_s \sim \mathrm{Gamma}(k_s I(s),\, k_s I(s)),$$
Here $R_s$ is the mean reproduction number, $w$ is the generation-interval distribution, and $\nu_s$ is the cohort-level infectiousness multiplier. Its variance shrinks as $1/(k_s I(s))$, which is the aggregate form of negative binomial offspring with dispersion $k_s$. The trajectories $\log R_s$ and $\log k_s$ follow smooth Gaussian-process or random-walk priors in continuous time.
A report $y_j$ covers an arbitrary interval $(a_j, b_j]$. It is modeled as the cases whose report time falls in that interval. Report time equals infection time plus a symptom-and-testing delay, plus a batching step. The batching step holds cases for a random number of days, with a day-of-week hazard, before releasing them. Because batching only moves cases between windows, the total over any long horizon is preserved. This mass conservation is what the inference exploits.
Observation noise beyond batching gets its own small dispersion parameter. It is identified only through transient, non-propagating deviations.
Inference uses a particle or Hamiltonian Monte Carlo sampler offline. For real-time use, we amortize the posterior over $(R_t, k_t)$ with a jump-ODE encoder in the style of NJ-ODE. The encoder is trained on simulations and updates its state at each irregular report. The amortized network is a speed device. Every claim is first checked against the exact posterior.
Research Program
The program has three parts. The first is an identifiability study on synthetic data. We ask under which combinations of epidemic size, $k$, reporting interval, and batching intensity the persistence signature separates the two variance sources. We also locate where it fails, and expect failure at very low incidence, where few cohorts carry information.
The second is the estimator itself, with an ablation that removes mass conservation. The third is application to historical data. There, ground truth is unavailable, so we test self-consistency instead. The $k_t$ estimate should not change when we deliberately degrade the reporting schedule of a clean series.
Baselines
The naive reference is IIH applied after forward-filling irregular reports onto a daily grid. This is the pipeline a practitioner would run today. It is the reference that Early Sepsis Prediction Using Ensemble Learning and the Neural CDE paper argue introduces bias under imputation.
The strongest heterogeneity competitor is IIH fit to weekly aggregated counts. Weekly bins remove weekend effects by construction, so it is the hardest baseline to beat on robustness, though it pays in temporal resolution. We also run EffDI with its own empirical delay distributions, because it is the existing method that does model delays.
The strongest continuous-time competitor is EARTH with its output replaced by a negative binomial head. This lets a flexible model try to absorb dispersion implicitly. For forecasting, EARTH reports RMSE 177.6 at a 10-day horizon on Australia-COVID, against 312.4 for the next best mechanistic baseline, EINN (EARTH). For $k_t$ estimation there is no reported result found on Lacuna for IIH or EffDI as a benchmark metric. Both papers report qualitative agreement with contact-tracing estimates rather than error tables. So the primary comparison is necessarily on simulations we construct.
The key ablation keeps our latent transmission model but treats each report as an independent negative binomial count with its own dispersion. This removes the mass-conserving batching layer, so any gain attributable to that layer shows up directly.
Fairness rules apply to every method. All methods see identical simulated and real series and the same generation-interval and delay distributions. Hyperparameters are tuned on a held-out simulation set with an equal budget of fits. The neural baselines get the same simulation corpus for training that our amortized encoder uses.
Beating the baselines means the following. Our estimator must have lower bias and better 90% interval coverage for $k_t$ than weekly-aggregated IIH under batching corruption. It must also be no worse under clean reporting. And it must detect changes in $k_t$ earlier than weekly binning permits, measured as detection delay in days after a simulated shift in dispersion.
Why not just…?
Why not just aggregate to weekly counts and run IIH? This is the real threat. If weekly IIH recovers $k_t$ as accurately as our model across the identifiability sweep, the contribution shrinks to temporal resolution, and we will report that. Two reasons suggest it will not suffice. First, batching delays often cross week boundaries, so week totals are still contaminated. Second, weekly bins cut the number of independent windows sevenfold, which widens $k_t$ intervals and delays detection of shifts. The detection-delay metric and the cross-boundary batching scenarios settle this question directly.
Why not just add a negative binomial observation-noise term to a renewal model? A free observation dispersion can absorb transmission variance as easily as reporting variance. Nothing in such a model distinguishes the two, so $k_t$ and the observation dispersion trade off along a ridge in the likelihood. Our ablation is essentially this model. If it identifies $k_t$ as well as the full model, then mass conservation is unnecessary and persistence alone suffices. That would itself be a useful, simpler finding.
Why not just use EARTH or a Neural CDE with a heavy-tailed output head? These models handle irregular timestamps well. However, their spline inputs smooth spikes, and their output noise is not causally linked to future incidence. A wide predictive distribution therefore cannot say whether a spike will echo one generation later. The evidence that such models misjudge tail risk is indirect, coming from the mean-field critiques in Pakkanen et al. and the IIH paper. We test it directly with a simple check. A post-hoc $k_t$ readout from the EARTH variant should show the same inflation under batching that EffDI does.
Why not just use contact-tracing line lists? Where line lists exist, they should be used, and we treat them as the external reference. Probabilistic tracing (Effectiveness of probabilistic contact tracing) works well inside agent-based simulators. But line lists are exactly what collapses during the surges when monitoring $k_t$ matters most.
Proposed Experiments
Synthetic ground truth comes from two sources. One is negative binomial branching simulations with prescribed $k_t$ paths: constant, stepped, and slowly drifting, with $k$ ranging from about 0.1 to 10. The other is the agent-based simulators OpenABM and Covasim, where heterogeneity arises from contact structure rather than being imposed. Each true series is corrupted by a grid of reporting processes: weekend pauses, Monday backlog dumps, random irregular intervals, uniform 60% thinning to match the IIH robustness test, and 40% random point removal to match EARTH's missingness test. Metrics are bias and coverage for $k_t$, detection delay for dispersion shifts, and a false-alarm rate. The false-alarm rate counts how often each method reports $k_t < 1$ when the truth is Poisson but reporting is batched. That is the failure EffDI warns about.
On real data, we use the IIH historical datasets: Hong Kong, Tianjin, MERS Korea, Ebola Sierra Leone, Georgia COVID-19, and South Africa Omicron. We also use the Austrian and JHU series used by EffDI. Each is degraded with synthetic batching, and we measure how much each method's $k_t$ estimate moves. For the static-$k$ outbreaks, we compare against the contact-tracing estimates that IIH already reports agreeing with. On Australia-COVID and the US influenza sets from EARTH, we check that forecasts from our model's posterior predictive stay competitive with the reported EARTH RMSE. The point there is to show we have not traded away mean accuracy, not to win on forecasting.
The decisive check is the Poisson-truth batching scenario. If our model keeps the false-alarm rate near nominal while IIH, EffDI, and the EARTH variant flag spurious superspreading, the core claim holds. If weekly-aggregated IIH also passes and matches our detection delay, we will conclude that the persistence argument adds little beyond binning. We will publish the identifiability map as the main result.
References
- Assessing the heterogeneity in the transmission of infectious diseases from time series of epidemiological data
- Characterizing superspreading events and age-specific infectiousness of SARS-CoV-2 transmission in Georgia, USA
- Pooled testing of traced contacts under superspreading dynamics
- Monitoring real-time transmission heterogeneity from incidence data
- Neural Ordinary Differential Equations
- Neural Controlled Differential Equations for Irregular Time Series
- Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filtering
- Epidemiology-Aware Neural ODE with Continuous Disease Transmission Graph
- Neural Ordinary Differential Equations for Modeling Epidemic Spreading
- Estimating the time-varying effective reproduction number via Cycle Threshold-based Transformer
- Early Sepsis Prediction Using Ensemble Learning with Features Extracted from LSTM Recurrent Neural Network
- Unifying incidence and prevalence under a time-varying general branching process
- Effectiveness of probabilistic contact tracing in epidemic containment: The role of superspreaders and transmission path reconstruction