Telling Diagnostic Upgrades from Real Disease Change in Global Surveillance Streams

When a country builds new laboratories or expands health coverage, its reported case counts rise. Today's forecasting models and drift detectors cannot tell whether that rise is an outbreak or simply more disease being found. This note proposes a detector that can. The key observation is that better diagnostic capacity raises recorded counts for many diagnosis-dependent conditions in that country at the same time, and roughly in proportion to how much each condition depends on diagnosis. A real change in disease, by contrast, usually affects one or a few conditions. If we monitor many disease streams per country together, the shared component of their change is evidence of a change in reporting, and the disease-specific remainder is evidence of a change in disease.

Concept drift is the machine-learning term for a change over time in the relationship a model has learned. In this setting it splits into two kinds. The first is ascertainment drift, a change in the probability that a true case gets recorded. The second is incidence drift, a change in true disease occurrence. The proposal treats the unmeasured diagnostic capacity of each country as a hidden variable estimated from how its disease streams move together. The concept-drift literature has so far assumed this variable is observed or absent.

Why this gap is real

The epidemiology literature documents the problem repeatedly but does not correct for it in a time-varying way. High-quality cancer registries cover only about one-third of the world's population, mostly in high-income nations, so pancreatic cancer burden is systematically underestimated where diagnostics are weak (Wong et al.). Similar under-ascertainment is reported for women's cancers (Li et al.; Yi et al.) and for ischemic heart disease mortality, where vital registration is incomplete (Zhi et al.). The pattern even produces paradoxes: musculoskeletal disorders look more common in high-income countries because those countries screen for them more (Liu et al.). These papers motivate the proposal. They establish that the reporting mechanism varies by place. Because capacity changes over time, it must also vary over time, but none of them models that.

The concept-drift literature has machinery for change detection, but each piece leaves a gap here.

Lagged-label methods such as CDCSDE assume true labels eventually arrive. In low-resource registries they never do.

The sequential classifier two-sample test gives exact false-alarm control. It does so only for shifts in the input distribution, not for shifts in how outcomes relate to inputs.

Regional concept drift with density-synchronized adaptation localizes drift to "regions," but these are nearest-neighbour clusters in feature space, not countries. It also assumes every point is fully observed.

The closest work is Context-Aware Drift Detection. It suppresses false alarms caused by harmless changes in an observed context variable by comparing deployment data only with reference data from similar contexts. Its kernel-based version, MMD-ADiTT, stays calibrated under such shifts, while standard maximum mean discrepancy (MMD, a kernel-based distance between two samples) flags drift with 100% certainty. Our problem is the same in spirit, since a capacity upgrade is exactly such a harmless context change, but different in substance. Here the context is unmeasured and has to be inferred.

Causal-Informed Contrastive Learning also treats drift as a hidden confounder. It removes the confounder inside an image representation and does not estimate it as an interpretable quantity. Its method is specific to images.

On the application side, the Variational-LSTM COVID forecaster trains on reported counts as ground truth. Its authors note that a negative fertility–spread correlation may reflect reporting differences, but the model offers no way to test this.

Core Claim

The capacity of each country to detect disease can be estimated without ground-truth labels from the co-movement of many diagnosis-dependent disease streams. Once estimated, it lets an online detector attribute each detected change to ascertainment or to incidence. Let $y_{c,d,t}$ be the reported rate of disease $d$ in country $c$ in year $t$. Let $\lambda_{c,d,t}$ be the true rate and $\pi_{c,d,t}$ the probability a true case is recorded. We model

$$\log y_{c,d,t} = \log \lambda_{c,d,t} + a_{c,d} + b_d\, g_{c,t} + \varepsilon_{c,d,t},$$

where:

  • $a_{c,d}$ is a fixed country-disease reporting offset;
  • $g_{c,t}$ is a latent diagnostic-capacity factor for country $c$;
  • $b_d \ge 0$ is how strongly disease $d$ depends on diagnosis (imaging- or biopsy-dependent cancers high, conditions with little diagnostic dependence near zero);
  • $\varepsilon$ is noise.

The model is identifiable under two assumptions. The first is that true incidence changes are sparse across diseases and not aligned with the loading vector $b$. The second is that a small set of "anchor" conditions has $b_d \approx 0$, which fixes the scale.

Detection then becomes two sequential tests on yearly increments. The first asks whether $\Delta g_{c,t}$ is non-zero; a yes is an ascertainment alarm. The second asks, for each disease, whether the residual $\Delta \log y_{c,d,t} - b_d\,\Delta g_{c,t}$ is non-zero; a yes is an incidence alarm. Adaptation follows from attribution. After an ascertainment alarm, the downstream forecaster keeps its learned disease dynamics and only re-scales the reporting offset. After an incidence alarm, the forecaster is updated for that disease alone. This contrasts with the flush-and-retrain logic in CDCSDE and the resampling in LDD-DSDA.

Research Program

We will build three things. The first is the factor model above, fitted by robust low-rank regression with a sparsity penalty on incidence increments, run over expanding yearly windows so that it is genuinely online. The second is a pair of sequential tests on the estimated increments. These will use a decaying significance level, because single-step error guarantees, as the sequential C2ST authors note, accumulate false alarms over long streams. The third is a simple downstream forecaster of true incidence that consumes the attribution.

The loadings $b_d$ can be set in two ways. One is a prior from clinical diagnostic dependence. The other is estimating them from the data, constrained by the anchors. We compare both to see how much the method depends on expert input.

The main data are the Global Burden of Disease (GBD) series used across the epidemiology papers above: 204 countries, 1990–2021, many conditions. We draw a panel of roughly two dozen conditions from those analysed in the file. On the high-dependence side are pancreatic, thyroid, breast, cervical, lung, and oral cancers, dementia, and depression. On the lower-dependence side are hip fractures and selected injury-related or mortality-anchored series.

One complication shapes the whole design. GBD values are themselves modelled estimates, already smoothed and partly imputed using Socio-Demographic Index (SDI, a composite of income, education and fertility) covariates. Real GBD series may therefore hide some ascertainment artefacts, and they cannot serve as ground truth for attribution. For this reason the decisive tests are semi-synthetic, and the real-data analyses are plausibility checks.

Baselines

The naive reference point is a forecaster trained on reported rates with no drift detection, refit yearly. This is the Variational-LSTM approach of Ibrahim et al. reduced to a per-country temporal model, since that paper's spatial branch needs daily data. It shows the cost of treating reported counts as truth. Beating it means lower error in forecasting true incidence on semi-synthetic data, not reported incidence.

The per-stream detector baseline runs the Incremental Kolmogorov-Smirnov test and MMD separately on each country-disease stream. Every alarm is classed as incidence drift, because these detectors have no notion of ascertainment. This represents what a practitioner would deploy off the shelf.

The strongest existing competitor is MMD-ADiTT from Context-Aware Drift Detection, with SDI and its yearly change as the observed context. A change in reported rates that SDI explains is treated as benign, playing the role of an ascertainment change; a change it does not explain is treated as incidence drift. This is the strongest competitor because it is the only published method built to separate harmless context-driven change from real change. Its weakness, if any, is that SDI is only a proxy for diagnostic capacity.

The only reported numbers come from that paper's synthetic setting. There MMD-ADiTT is calibrated under purely harmless shift (uniform p-values, against 100% false alarms for standard MMD), and it reaches an area under the ROC curve (AUC) of 0.97 for mean shifts at $N=512$. There is no reported result found on Lacuna for any drift detector on GBD or other disease-surveillance panels, so we will establish the number ourselves.

A static adjustment baseline regresses reported rates on SDI and health expenditure tiers. It flags drift when residuals exceed control limits. This is the standard epidemiological correction.

Two ablations isolate the key ingredient. The first fits our model to one disease at a time, which removes cross-disease pooling. The second removes the anchor conditions.

All methods see the same country-years. All detectors are tuned to the same nominal false-alarm rate on a held-out set of pre-2000 years. Each method receives the same hyperparameter search budget, about fifty configurations.

Beating the strongest competitor means two things. In the semi-synthetic regime where capacity upgrades are decoupled from SDI, our method should show higher AUC for attributing a change to ascertainment versus incidence. It should also show a lower false-incidence-alarm rate under pure ascertainment shocks, at matched detection delay.

Why not just…?

Why not just adjust for SDI as a covariate? SDI changes slowly and smoothly. Laboratory openings, screening programmes and registry launches are abrupt and country-specific. The GBD analyses themselves show large dynamics within a single SDI tier: low-SDI regions carry the highest chronic kidney disease burden while high-SDI regions rise fastest (Wang et al.). A static adjustment cannot follow step changes in capacity. The semi-synthetic experiment injects step-shaped upgrades and measures exactly this.

Why not just run MMD-ADiTT with SDI or health spending as the context? This is the real threat. If observed proxies track diagnostic capacity closely, conditioning on them is simpler and comes with the calibration guarantees of Cobb and Van Looveren. We settle the question with a dial in the simulator that sets the correlation between injected capacity upgrades and SDI change, from fully coupled to independent. We expect MMD-ADiTT to match us when coupling is high and to fail as coupling falls. If it matches us across the whole range, or if real GBD residuals show that estimated capacity is almost a function of SDI, we will report that the latent-factor approach is unnecessary.

Why not just use mortality, which is hard to miss, as the ground truth? Mortality ascertainment is itself incomplete where vital registration and autopsy coverage are weak, as the ischemic heart disease analysis documents (Zhi et al.). Mortality also responds to treatment access, which changes alongside diagnostic capacity. We still use mortality-anchored series among the low-loading anchors, and we test how sensitive attribution is to dropping them.

Why not just treat low-resource countries as a target domain and apply unsupervised domain adaptation? Unsupervised domain adaptation means methods that align a labeled source with an unlabeled target. On non-image tasks in WILDS 2.0, none of these methods gave a significant gain over plain empirical risk minimization, and our data are small tabular yearly panels. More fundamentally, aligning feature distributions across countries would erase real differences in disease burden along with reporting differences. That is the opposite of what we want.

Proposed Experiments

Semi-synthetic attribution test (decisive). We treat smoothed GBD trajectories as true incidence. We generate reported series by applying country-specific detection probabilities with step-shaped capacity upgrades, loaded onto diseases through clinically motivated $b_d$. Independently, we inject disease-specific incidence shocks of varying size, sometimes in the same year as an upgrade. Ground truth is known, so we measure:

  • attribution AUC;
  • false-incidence-alarm rate under pure upgrades;
  • detection delay in years;
  • error of the downstream forecaster against true incidence.

These are reported across the SDI-coupling dial and across panel sizes from three to twenty-four diseases. The pooling argument predicts that attribution improves as more diseases are added. If it does not, the core mechanism is wrong.

Assumption stress test. We break the assumptions on purpose. Incidence shocks are made aligned with $b$, as when a true epidemic hits mainly imaging-dependent cancers, and anchors are given non-zero loadings. This shows how the method degrades and reports the breakdown point honestly.

Real-data plausibility checks. On real GBD series, the estimated capacity factor should be higher in high-SDI countries. This would reproduce the documented reporting paradox for musculoskeletal disorders. Its yearly changes should also explain a meaningful share of the co-movement among cancers in countries where reported cancer rates rose together. Because GBD values are modelled estimates, we present these checks as consistency evidence, not as validation.

A null result is informative either way. A flat capacity factor on real data would suggest that GBD's own modelling has already removed the ascertainment artefacts the method looks for. It would tell us the method belongs upstream, on raw registry and surveillance feeds.