Hospital surge decisions depend on one question: will admissions next week exceed the beds we have? The data needed to answer it arrive late. Each week's count keeps growing for weeks as reports trickle in, so the most recent part of any epidemic curve looks deceptively flat. Forecasters respond in three ways. They "nowcast" the eventual counts, they train models on log-transformed counts because counts are skewed and noisy in proportion to their size, and they average several models together. This proposal argues that the usual way of combining these three steps biases alarms downward in a predictable direction. When models are averaged on the log scale and the result is exponentiated back, the estimate is too low by an amount that grows with uncertainty. Uncertainty is highest in the most recent, least complete weeks, which are exactly the weeks a surge decision is based on.
The fix is simple to state. A capacity decision only needs the probability that eventual admissions exceed capacity $C$. That probability does not change under any monotone transform such as the logarithm, because $P(Y > C) = P(\log Y > \log C)$. If each base model reports its probability of crossing the threshold, the models can be combined at that point, with weights that depend on how complete the current week's reports are, and the bias from transforming back to raw counts disappears. Each model can still be trained on whatever target scale suits it. We propose to build this threshold-level, delay-aware ensemble and to judge it by clinical net benefit for surge alarms rather than by squared error. We will test whether lower RMSE on nowcasts actually turns into better decisions.
Core Claim
Two claims should hold together. First, a log-scale mean ensemble that is naively exponentiated under-triggers surge alarms, and the shortfall grows with how incomplete the most recent weeks' reports are. Second, pooling exceedance probabilities with weights conditioned on reporting completeness gives higher net benefit across the range of cost ratios a hospital would plausibly use. A good result would not just say "ensembles help." It would show that the space in which models are combined matters most at the right edge of the curve, and that the right space for a threshold decision is the threshold itself.
The mechanism is easy to see under a lognormal approximation. If a model's log-scale prediction for eventual week-$t$ admissions has mean $\mu_t$ and variance $\sigma_t^2$, the raw-scale mean is
$$\mathbb{E}[Y_t] = \exp\!\left(\mu_t + \tfrac{1}{2}\sigma_t^2\right),$$
whereas the naive back-transform reports $\exp(\mu_t)$. That estimate falls short by a factor of $\exp(\sigma_t^2/2)$. Because $\sigma_t^2$ grows as the fraction of eventual reports already received shrinks, the error is concentrated in the weeks that drive decisions. The research file names this retransformation bias, the consequence of Jensen's inequality, as a known limitation of log targets. It also notes that averaging predictions made on differently transformed scales is mathematically inconsistent. As far as the file shows, no one has traced either problem through a nowcasting pipeline built on delayed reports.
Position Relative to Prior Work
The closest work falls into three groups. Each covers part of the question.
The first group nowcasts delayed counts with one carefully designed model. Gaussian process nowcasting models the full reporting triangle, meaning counts indexed by event week and reporting delay, with a negative binomial likelihood and two-dimensional Gaussian process kernels on Brazil's SIVEP-Gripe data. It shows that uncorrected data suggested a shrinking epidemic ($R_t<1$) when the epidemic was actually growing. This paper motivates our work, since it shows that latency changes decisions, and it supplies our strongest base model. It does not use transformed targets, does not ensemble, and does not evaluate any decision threshold. EpiFlow denoises wastewater signals and forecasts Virginia hospitalizations with rolling vector autoregression, and it remains robust under simulated 1–2 week delays. It too relies on a single linear model scored by interval coverage.
The second group ensembles under latency but either avoids continuous targets or does not combine models at all. The prospective early warning system raises an alarm when a fraction of search-term and neighbouring-state proxies all show exponential growth. This is a vote over binary signals. It is the strongest existing alarm system in the file, and we treat it as a baseline rather than a method to reuse. The Korean symptom-search study compares 20 forecasting architectures one at a time and leaves multi-model ensembling for future work. Its finding that each architecture prefers different keywords is evidence that the members would make different errors, which is what an ensemble needs.
The third group constrains the design. The When and How of Target Variable Transformations shows that linear models gain 11–63% median relative squared error from target transformation, while gradient boosted trees are indifferent or slightly hurt. An ensemble of mixed model families therefore cannot share one target scale without disadvantaging some members. Our design responds by letting each member keep its preferred scale and combining them only at the threshold. On the clinical side, the utility literature repeatedly finds that average-case accuracy does not establish whether a model is safe to act on. Examples include the cirrhosis classifiers of Ma et al. and the treatment-rule bounds of Nonparametric Bounds for Evaluating the Clinical Utility of Treatment Rules. We borrow that stance for surveillance, which so far has been scored only by RMSE, CRPS and interval scores. We do not use their methods. Finally, Bias Delayed is Bias Denied? shows that reports which arrive late are systematically different from those that arrive early. This rules out treating delay as missing-at-random imputation and supports modelling the reporting triangle directly.
Research Program
We will reconstruct real-time "vintages," meaning the data exactly as they stood on each historical date, and run every method against those vintages, never against later corrected data. SIVEP-Gripe records report dates alongside onset and hospitalization dates, so true vintages can be built. For the Virginia wastewater–hospitalization data and the Korean sentinel data, the file records no revision history. There we will impose delay distributions estimated from SIVEP, including the longer delays seen during peaks, and label those results as semi-synthetic.
The base models will cover different model families and target scales. They are the reporting-triangle Gaussian process (negative binomial, raw counts), gradient boosted quantile regression on raw counts, a linear autoregressive model with leading signals on log counts, and a recurrent or DLinear network on log counts with search or wastewater inputs where available. Each model must output a full predictive distribution for eventual week-$t$ admissions. From that distribution we read off $p_{k,t} = P_k(Y_t > C)$ for member $k$.
We combine the members with completeness-conditioned weights:
$$\hat p_t = \sum_k w_k(c_t)\, p_{k,t}, \qquad \sum_k w_k(c_t)=1,$$
where $c_t$ is the estimated fraction of week $t$'s eventual count already reported, read from historical delay curves. The weights $w_k(\cdot)$ form a softmax over a small spline in $c_t$, fitted on past vintages. The fitting objective is the Brier score for the event $Y_t > C$. This scoring rule rewards honest exceedance probabilities, and averaged over cost ratios it corresponds to net benefit. We expect the triangle model to dominate when $c_t$ is high and the leading-signal models to gain weight when $c_t$ is low. That pattern is checkable, and it gives the method an interpretation.
The capacity $C$ is set per region from historical capacity proxies, for example a high percentile of past weekly admissions, with sensitivity analysis over several percentiles. We evaluate alarms that fire when $\hat p_t$ exceeds a probability threshold $p^\ast$. Clinical utility is measured with net benefit,
$$\mathrm{NB}(p^\ast) = \frac{\mathrm{TP}}{N} - \frac{\mathrm{FP}}{N}\cdot\frac{p^\ast}{1-p^\ast},$$
where TP and FP are true and false alarms over $N$ region-weeks. Here $p^\ast/(1-p^\ast)$ is the cost of a false alarm relative to a missed surge. We trace net benefit across that ratio as a decision curve.
Baselines
The naive reference points are the uncorrected latest reported count, and a historical seasonal average for the same week. The seasonal average is a serious reference: Heuton et al. found that historical averages beat deep networks when latency is ignored. Both will be thresholded at $C$ and scored with the same decision curves. Beating them means positive net benefit above both across the central range of cost ratios.
The strongest existing nowcaster is the reporting-triangle Gaussian process of Hawryluk et al.. It is the only method in the file that models delay explicitly and gives calibrated uncertainty, and it outperformed or matched NobBS on RMSE and CRPS. The file reports no scalar RMSE or CRPS on SIVEP-Gripe; no reported result was found on Lacuna for these. We will rerun it on identical vintages, with the same training windows and the same budget for tuning kernel hyperparameters as the ensemble's weight fitting. Our ensemble beats it if it shows higher net benefit at matched cost ratios and a lower threshold Brier score, with the difference holding under a block bootstrap over seasons.
For alarm detection on U.S. data, the strongest competitor is the early warning system's proxy vote. It reports 98.0% onset sensitivity, 5-week average lead time, and PPV above 82% in the 2024–2025 season. We will reimplement its voting rule on the same proxies and seasons. Because it detects outbreak onset rather than capacity exceedance, we score both methods on our capacity task and on onset sensitivity and PPV. Matching its sensitivity at higher PPV counts as a win.
For point accuracy on the Korean data, the reference numbers are rhinovirus RMSE of 82.3 for DLinear with "sputum" against 95.7 for patient history, and 62.9 for GRU with "rhinitis" against 64.9, from the Korean study. On Virginia, EpiFlow's VAR-dn reports a 20 percentage point gain in 95% coverage during surges. We use these to confirm that our base models are competitive before ensembling.
Three ablations isolate the key ingredient. The first averages members on the log scale and exponentiates naively. The second does the same but applies the variance correction $\exp(\mu+\sigma^2/2)$ before thresholding the mean. The third is an equal-weight linear pool of exceedance probabilities with no conditioning on completeness. All three use the same members and the same fitting data.
Why not just…?
Why not just apply the variance correction to the log-scale ensemble mean? It removes the bias in the mean, but a capacity decision should compare a probability with a cost ratio, not compare a point estimate with $C$. A corrected mean that sits just below $C$ says nothing about whether the chance of exceeding it is 10% or 45%. The correction also assumes a lognormal shape that overdispersed, back-filled counts may not have. The smeared-mean ablation tests this directly. If it matches our net benefit, the threshold-pooling idea adds little and we will report that.
Why not just use the single reporting-triangle Gaussian process, which already handles skew through its negative binomial likelihood? This is the main threat. That model avoids transformation bias altogether, and if it dominates at every completeness level the ensemble is unnecessary. Our expectation is that it has nothing to lean on in the newest weeks, when almost nothing has been reported, while search and wastewater signals do. The fitted weight curves $w_k(c_t)$ and the net benefit comparison restricted to low-completeness weeks will settle the question.
Why not just pool probabilities with equal weights? Equal weights are a hard baseline for ensembles, and they may capture most of the gain. The equal-weight ablation, split by completeness level, tells us whether conditioning on delay earns its added complexity. If it does not, the contribution reduces to "pool at the threshold," which is still a useful finding.
Why not just use gradient boosted trees on raw counts and skip transformations? Nuyts and Davis show trees barely care about target scale, so a tree-only system avoids the problem. Trees cannot extrapolate beyond counts seen in training, however, and an unprecedented surge is exactly the event to be caught. The raw-scale tree is one of our members and will also be scored alone.
Decisive Checks
The first check establishes the mechanism. On SIVEP vintages, we plot the naive log-mean ensemble's shortfall in exceedance probability against completeness $c_t$. The claim fails if the shortfall does not grow as completeness falls. The second check tests the main claim: decision curves show completeness-conditioned threshold pooling above the Gaussian process, the equal-weight pool, and the smeared mean over a central band of cost ratios, with season-block bootstrap intervals that exclude zero. The third check tests the clinical-utility argument by ranking all methods by RMSE and by net benefit. If the rankings disagree for some methods, surveillance evaluation that relies only on squared error is choosing the wrong models. The final check repeats the comparison under longer delays during peaks on the semi-synthetic Virginia and Korean data, where delays stretch precisely when hospitals are full.
References
- Gaussian process nowcasting: application to COVID-19 mortality reporting
- EpiFlow: A framework for improving the utility of wastewater signals for disease forecasting
- A Prospective Real-time Early Warning System to Anticipate Onsets and Peaks of Respiratory Diseases Outbreaks at the State Level in the U.S. A Transfer Learning Approach Leveraging Digital Traces
- Can Symptom Search Trends Forecast Respiratory Virus Patient Counts? A Rhinovirus Case Study in Korean Syndromic Surveillance
- The When and How of Target Variable Transformations
- Relationship between symptoms of traditional Chinese medicine and indicator of western medicine about liver cirrhosis
- Nonparametric Bounds for Evaluating the Clinical Utility of Treatment Rules
- Bias Delayed is Bias Denied? Assessing the Effect of Reporting Delays on Disparity Assessments