Most disease records come from passive surveillance, meaning a case appears in the data only if the person got sick enough to seek care, could reach a clinic, was tested, and had the result reported. A machine-learning model trained on such records learns two things at once: who is infected, and who is likely to be seen. Recursive feature elimination (RFE) makes this worse. RFE is a common way to shrink a large set of inputs: fit a model, drop the least useful input, refit, and repeat. Under passive labels it keeps whatever predicts being reported, such as clinic density, insurance proxies, or features of affluent neighborhoods. It can also drop features that track infection in places where few infections get recorded.
We propose a small change that targets this failure. The model is still trained on the plentiful passive labels. At every pruning step, though, a feature's usefulness is judged by how much it helps predict a scarce reference signal that does not depend on care-seeking. Wastewater viral load, serology surveys, and household surveys are such signals. We call this anchor-scored RFE. The claim is that this recovers the features that track infection, and narrows the gap in prediction error between poor and wealthy areas. It should do so better than three simpler routes: reweighting the samples, training on the reference signal alone, or tuning RFE's stopping point to the worst-off subgroup.
Core Claim
Write $y$ for true infection, $s \in \{0,1\}$ for whether an infection is ascertained (seen, tested, and reported), and $a(x) = P(s=1 \mid x)$ for the ascertainment probability given features $x$. A passive record is $y_{\text{obs}} = y \cdot s$. If $s$ depends on $x$ but not further on $y$, then
$$ E[y_{\text{obs}} \mid x] = a(x)\, P(y=1 \mid x). $$
Any feature that moves only $a(x)$ therefore has real predictive value for $y_{\text{obs}}$ and none for $y$. Every standard RFE importance score is computed against $y_{\text{obs}}$, so it cannot tell the two kinds of feature apart. This includes linear weights, impurity and permutation importance, and SHAP (an attribution method that splits each prediction into per-feature contributions). Pruning also compounds the problem. Once an infection-tracking feature is gone, the access proxies that remain absorb its weight and look even more important at the next step. This is the same weight-absorption behavior that makes RFE good at handling redundant features.
The fix is to keep the fitted model and change only the yardstick. Let $f_d$ be the model fitted on $y_{\text{obs}}$ with the current $d$ features. Let $A$ be an anchor: a measurement on a subset of units whose generation does not pass through the care-seeking filter. We score feature $j$ by its permutation importance for predicting $A$ from $f_d$'s output, after a one-dimensional calibration of $f_d$'s scale to $A$. We prune the lowest-scoring feature and refit on $y_{\text{obs}}$. The anchor never has to be large enough to train a model. It only has to be large enough to rank features, which is a much easier statistical task. This is why the method suits settings where serology covers 10 sites or a survey runs once.
Relation to Prior Work
The motivating evidence comes from surveillance epidemiology. Socioeconomic bias in influenza surveillance fit lasso-regularized additive models on ILINet, BioSense and Google Flu Trends in Dallas-Fort Worth. It found that out-of-sample error was highest in the poorest zip-code quartile ($P<0.0001$), and it ruled out epidemic asynchrony as the cause. Its lasso selects inputs purely by fit to the reported target. We keep its data and stratification, but replace selection-by-fit with selection-by-anchor and test whether the quartile gap shrinks.
Estimating incidence of sexually transmissible infections in Australia showed that screening intensity can reverse an apparent sex difference in chlamydia incidence, and that naive notification-to-test ratios got the trend direction wrong. Its probability-tree structure is the template for our simulator.
Two other papers correct for under-reporting but select nothing. Accurately Estimating Unreported Infections using Information Theory calibrates a low-dimensional epidemic model against CDC serology using minimum description length, the principle of choosing the model that gives the shortest combined encoding of model plus data. On the origin and timing of Zika virus introduction in Brazil rescales clinical counts by an assumed subclinical fraction. Both treat the reporting filter as a scalar or a few parameters. We treat it as something that leaks into a high-dimensional feature matrix, which those methods do not address. We reuse their serology data as an anchor and their reporting-rate estimates as an inverse-weighting baseline.
The anchors themselves come from studies showing that non-clinical streams bypass the filter. Beyond clinical data: the role of sewage monitoring in tracking pandemic trends of SARS-CoV-2 found sewage RNA correlated with clinical positivity at Spearman $\rho=0.97$, with a lead of about one week. Wastewater sequencing reveals the genomic landscape of Influenza A virus in Switzerland and A population-based survey of the prevalence of self-reported acute gastrointestinal illness in Zhejiang Province, China add two more. The Zhejiang survey is telling: only 3.4% of ill people gave a stool sample.
On the method side, the RFE applications in the file are all fitted and scored downstream of the clinical filter. These are SHAP-RFECV on BIMCV chest X-rays in Explainable Machine Learning for COVID-19 Pneumonia Classification With Texture-Based Features Extraction in Chest Radiography and RFE plus random forest for candidemia in A machine learning model for early candidemia prediction in the intensive care unit: Clinical application. We borrow their pipelines unchanged except for the scoring target.
The closest methodological neighbor is Recursive Causal Structure Learning in the Presence of Latent Variables and Selection Bias. It removes variables one at a time while learning a causal graph under selection bias. It is unsupervised, assumes the selection-induced subgraph is chordal, and needs no anchor. We address the supervised predictive question and make no graphical assumption about the filter. In exchange, we require an external anchor.
Research Program
The work has three parts.
The first is a semi-synthetic simulator patterned on the Australian probability tree and the Dallas-Fort Worth design. Simulated zip codes carry real census-style covariates. Infection depends on one known feature set and ascertainment on a partly overlapping known set, with access falling as poverty rises. A noisy anchor is observed in a small random fraction of units. Because the features that truly drive infection and those that drive ascertainment are known, we can measure exactly what each selection method keeps.
The second is the algorithm and its variants. These include anchor-scored RFE with linear, random forest and gradient-boosted estimators. They also include a stopping rule, chosen by cross-validation, that uses anchor error rather than passive-label error. Finally, we add a bootstrap over anchor units to report how stable the ranking is when the anchor is small.
The third is real-data tests on three settings in the file where a passive label and an anchor coexist:
- Dallas-Fort Worth influenza. Lagged ILINet, BioSense, Google Flu Trends and zip-level census covariates give several hundred features. ILINet visits serve as the passive target and inpatient hospitalizations as the more access-robust anchor.
- US 2020 COVID-19. New York Times case counts are the passive target and the 10-site CDC serology is the anchor.
- Yantai. Sentinel-hospital positivity is the passive target and wastewater RNA is the anchor.
Baselines
The naive reference is standard RFE with cross-validation, scored and stopped on the passive label. This is the pipeline of the radiomics and candidemia papers. Within each dataset we run it with the same estimator, the same feature matrix, and the same hyperparameter grid as every other method. A second naive reference is no selection at all, which separates the effect of pruning from the effect of the scoring target.
The strongest existing method on the influenza question is the lasso-penalized additive model of Socioeconomic bias in influenza surveillance. That paper reports only the significance of the quartile disparity ($P<0.0001$) and the ordering of quartiles. It gives no per-quartile error values, so there is no reported result found on Lacuna for the lasso model's per-quartile normalized out-of-sample RMSE. We will refit it ourselves on the same data splits and treat our reproduction as the reference number. For context, the only reported numbers for RFE-based pipelines in the file are AUROC 0.87 internal and 0.83 external for RFE plus random forest in A machine learning model for early candidemia prediction in the intensive care unit: Clinical application, and accuracy 0.82 for SHAP-RFECV plus XGBoost in Explainable Machine Learning for COVID-19 Pneumonia Classification With Texture-Based Features Extraction in Chest Radiography. Neither is evaluated against an anchor, so neither is directly comparable.
Three further baselines are the real competitors:
- Inverse-probability-weighted RFE. Each passive record is reweighted by an estimated $1/a(x)$, using reporting rates in the style of MDLInfer for COVID and a poverty-indexed access model for influenza.
- Anchor-only RFE. Both fitting and scoring use the anchor alone.
- Worst-quartile RFE. Standard RFE whose stopping point is chosen on the worst poverty quartile's passive-label error.
The key ablation swaps only the scoring target, from anchor to passive label, while keeping fitting, estimator and stopping rule fixed. Any gain then comes from the yardstick alone.
Beating these baselines means three things. Prediction error against the anchor must be lower on held-out regions. The gap in that error between the poorest and richest quartiles must be smaller. In simulation, recall of the true infection features must be higher and retention of pure ascertainment features lower. Each must hold with bootstrap intervals that exclude zero, and without worse error on the passive label in the richest quartile.
Why not just…?
Why not just reweight samples by inverse ascertainment probability and run ordinary RFE? This is the most serious threat, and when $a(x)$ is known it should be about as good as our method. The problem is that in passive surveillance $a(x)$ is exactly what is unknown. The existing estimates are a single subclinical fraction for Zika (On the origin and timing of Zika virus introduction in Brazil) or a constant reporting rate (Accurately Estimating Unreported Infections using Information Theory). Neither varies with the features RFE is ranking. The simulator settles the question directly: we run inverse weighting with the true $a(x)$, with a misspecified $a(x)$, and with an estimated one. If the estimated-weight version matches anchor scoring, we will report that the cheaper method suffices.
Why not just train on the unbiased stream and skip passive data? Anchors are scarce. CDC serology covers 10 sites, and the Yantai wastewater set is 375 samples from 7 plants. A model fitted only on them overfits a several-hundred-feature matrix. Anchor-only RFE is a baseline, and we expect it to lose on held-out regions while winning on in-sample anchor fit. If it does not lose, the passive data adds nothing and the proposal fails.
Why not just tune ordinary RFE to the worst-off subgroup? This needs the access strata to be known and correctly drawn. Poverty quartiles are one proxy among many, and the Australian chlamydia case shows that ascertainment can also vary by sex and by screening program. Worst-quartile stopping also changes only when to stop, not which features are ranked low. In simulation we will make ascertainment depend on a variable not used to define the strata. We predict that worst-quartile RFE then fails while anchor scoring does not.
Proposed Experiments
Simulation. The decisive check is whether anchor scoring retains the true infection features and drops the pure ascertainment features as the access gradient steepens, while passive-scored RFE does the reverse. We vary the anchor fraction from 1% to 20% and its noise level to find the point at which the anchor becomes too small to rank features reliably. That breakdown point is the method's practical limit, and we will report it.
Dallas-Fort Worth. We use leave-one-season-out validation and report normalized out-of-sample RMSE against hospitalizations by poverty quartile, with the original paper's permutation test on the quartile gap. The method succeeds if the gap shrinks relative to our refit of the lasso model.
COVID-19. We hold out serology sites, then compare inferred infection trajectories with the serology confidence intervals that MDLInfer was validated against. For this check we feed each method's selected features into a simple regression rather than a compartmental epidemic model, so the comparison isolates feature selection.
Yantai. We test whether features selected by anchor scoring preserve the roughly one-week wastewater lead over clinical positivity that the original study reports.
A negative result is informative in two ways. If no setting shows a narrower quartile gap, ascertainment leakage into feature rankings is not the bottleneck. If estimated-weight inverse weighting matches anchor scoring everywhere, the cheaper correction is enough.
References
- Socioeconomic bias in influenza surveillance
- Estimating incidence of sexually transmissible infections in Australia
- Accurately Estimating Unreported Infections using Information Theory
- On the origin and timing of Zika virus introduction in Brazil
- Beyond clinical data: the role of sewage monitoring in tracking pandemic trends of SARS-CoV-2
- Characterizing influenza A virus lineages and clinically relevant mutations through high-coverage wastewater sequencing
- A population-based survey of the prevalence of self-reported acute gastrointestinal illness in Zhejiang Province, China
- Explainable Machine Learning for COVID-19 Pneumonia Classification With Texture-Based Features Extraction in Chest Radiography
- A machine learning model for early candidemia prediction in the intensive care unit: Clinical application
- Recursive Causal Structure Learning in the Presence of Latent Variables and Selection Bias