Forecasting Swine Outbreaks When the Labels Come From Who Gets Tested

Machine-learning models now forecast which pig farms will have an outbreak of porcine reproductive and respiratory syndrome virus (PRRSV), porcine epidemic diarrhea virus (PEDV), or respiratory pathogens weeks ahead. Their training labels, however, are not a record of which farms were infected. They are a record of which farms were infected and tested. High-value breeding farms are tested routinely. Finishing farms, where pigs are grown to market weight, are tested mostly when animals are visibly sick. Healthy finishing farms are almost never tested, so modelers invent "assumed negatives" from low mortality and good growth.

A model trained on such labels can score well by learning where testing happens and which farms look healthy on paper, rather than which farms are at risk of infection. We propose to treat the farm's surveillance regime as the "group" in a multi-stage bias mitigation pipeline, correcting at the data, training, and output stages. The part the field lacks is an evaluation where the true infection status is known, and we will build one. With it we can measure how much of today's reported accuracy is inflated by the testing process, and which correction stages actually remove that inflation.

Core Claim

Current swine outbreak forecasters are partly predicting the surveillance process. The file contains two concrete symptoms.

The first comes from Infection prediction in swine populations with machine learning, which built daily infection forecasts on two production networks. It lists historical testing rate among the most important predictors. That feature is a direct proxy for how much a farm is observed.

The second comes from Machine Learning Framework for Porcine Reproductive and Respiratory Syndrome Outbreak Forecasting, which forecast PRRSV across 3,770 finishing periods. It defines negatives by low mortality and high average daily gain, yet historical mortality is also one of its top predictors. The label definition therefore leaks into the inputs. Some of the reported discrimination may be the model recovering its own labeling rule. That paper also reports unstable feature selection in its best tree models, which is what one expects if different data splits expose different surveillance artifacts.

Our claim has three parts. First, the gap between accuracy on surveillance-generated labels and accuracy on ground-truth infection status is measurable and large for PRRSV and PEDV. Second, no single-stage fix closes it: reweighting fails where finishing farms are almost never tested, and threshold tuning cannot repair features that encode testing behavior. Third, a pipeline has to correct the labels, the learned score, and the per-regime calibration together. We state this as a hypothesis to be falsified, not a premise.

Role of Prior Work

The swine papers above motivate the problem. They supply the cohorts, the features, and the reported numbers, and between them they expose the label and feature leakage.

Two bias-mitigation papers supply method and caution.

Evaluating the Impact of Medical Image Reconstruction on Downstream AI Fairness and Performance showed three things in medical imaging. Upstream reconstruction models leave downstream accuracy nearly unchanged (CheXpert AUROC moves about 1% despite PSNR, a pixel-fidelity score, dropping up to 26%). Those same models shift sensitivity and specificity by about 5% across sex and age. Mitigation applied only upstream helped little and inconsistently across datasets. We borrow the lesson that aggregate accuracy hides stage-induced disparity. We also borrow its warning that mitigation effects are dataset-dependent, which is why we test across two production systems.

Bot Meets Shortcut: How Can LLMs Aid in Handling Unknown Invariance OOD Scenarios? combined three levels of correction against shortcut features in bot detection: counterfactual rewriting, dataset balancing, and contrastive training. This recovered an average 56% of performance lost to shortcuts. We reuse the idea of pairing a data-level correction with a training-level invariance objective. We do not use language-model rewriting, because counterfactual farm-days cannot be generated by editing text.

Neither paper faced labels that are missing in a pattern set by the very features the model sees. That is the new ingredient here.

Research Program

The central artifact is a ground-truth evaluation built by "surveillance thinning." Breeding (sow) farms under routine PRRSV and PEDV monitoring have close to complete ascertainment, meaning nearly every infection is detected. We construct thinned copies of their records in three steps. First, we drop routine tests. Second, we keep a test only when a clinical trigger fires, such as a mortality spike or a feed-intake drop, calibrated to match the testing frequency observed on finishing farms. Third, we apply the assumed-negative heuristic of the UC Davis paper.

Models train on thinned labels and are scored against the unthinned ones. Because thinning happens within the same farms, the comparison isolates the surveillance mechanism from biological differences between farm types.

We complement this with a fully synthetic setting. We simulate network transmission over the real pig-movement graph, using the 20 km movement radius and 60-day climate windows the UC Davis paper found predictive. We then pass the simulated infections through a testing model that depends on farm tier. Here the true infection status is known everywhere, including on finishing farms.

The mitigation pipeline has three stages, each aimed at a distinct failure.

At the data stage, we fit a model of the probability that a farm-period is tested, given its features and tier. We replace heuristic assumed negatives with soft labels from a positive-unlabeled model, a classifier that learns from confirmed positives and unlabeled cases, where the unlabeled cases are weighted by that testing probability. We clip weights where testing is nearly absent rather than letting them explode.

At the training stage, we remove the label-defining variables (mortality and growth indices within the labeling window) from the inputs. We add a penalty on the variance of the training loss across surveillance regimes: routine-tested breeding, reactive-tested finishing, and the thinned copies. The penalty discourages scores whose relationship to the label changes with how intensely a farm is watched.

At the output stage, we recalibrate risk scores separately for each tier. We use a small audited subset (the unthinned breeding records, plus any finishing-farm periods with scheduled testing) to estimate tier-specific label-noise rates.

Baselines

The naive reference point is a gradient-boosting model on the full feature set, historical testing rate included, trained on the heuristic labels. This is the standard recipe of both swine papers. The strongest existing numbers are Halev et al.'s balanced accuracies from Infection prediction in swine populations with machine learning: 85.3% for any disease in System A, and 74.8% for Mycoplasma hyopneumoniae, 72.8% for influenza A, and 58.0% for both PRRSV and PEDV in System B. The UC Davis paper reports AUC-ROC of 0.86–0.88 with Random Forest and Gradient Boosting. All of these are measured against surveillance-generated labels. We found no reported result on Lacuna for any swine forecaster evaluated against ground-truth or fully ascertained infection status, so we will reproduce both pipelines on the same cohorts to obtain that number.

The second baseline is Halev et al.'s autoencoder multi-layer perceptron, the one existing method designed for distribution shift between production systems. It is the strongest learned competitor, so it receives the same hyperparameter search budget as our pipeline.

The third baseline is the strongest simple epidemiological fix. It combines inverse-probability-of-testing weighting with removal of the testing-rate feature, tuned identically.

All methods share the same splits. Splits are temporal (train on earlier years, test on later years) and farm-disjoint. All methods also share the same feature pool and equal tuning trials.

We count the pipeline as beating these baselines only under three conditions. Its ground-truth AUC and balanced accuracy on finishing-tier farms must exceed every baseline across seeds and both systems, with non-overlapping bootstrap intervals. Its per-tier calibration error must be lower. Its advantage must come with a smaller gap between biased-label and ground-truth scores.

Ablations remove one stage at a time. They also include a version that keeps all stages but restores the label-defining features, which tests the leakage claim directly.

Why not just…?

Why not just drop the testing-rate feature? This is the most serious threat, and we include it as a baseline rather than dismiss it. It removes one proxy, but it leaves the labels unchanged: assumed negatives still come from mortality and growth, and farm density and inventory still correlate with testing intensity. The thinning benchmark settles the question. If dropping the feature closes most of the gap between biased-label and ground-truth AUC, the in-processing and post-processing stages are unnecessary, and we will report that.

Why not just reweight by the inverse probability of being tested? Reweighting is correct when every kind of farm has some chance of being tested. On finishing farms that chance is close to zero for healthy periods, so the weights become enormous and the estimates noisy. The reconstruction study found that mitigation at a single early stage was weak and inconsistent downstream (Evaluating the Impact of Medical Image Reconstruction on Downstream AI Fairness and Performance), which is suggestive but not decisive in this domain. We will vary the testing rate in the synthetic setting to find where reweighting breaks down.

Why not just set a lower alert threshold for finishing farms? Per-tier thresholds change who gets flagged, but not the ranking within a tier. If the score ranks farms by how sick they look on paper, no threshold recovers silent low-pathogenicity outbreaks. Ground-truth AUC within the finishing tier is invariant to thresholds, so it separates this baseline from the pipeline.

Why not rely on the mutual-information feature selection or the shift-handling autoencoder already published? The UC Davis authors report unstable feature selection in their best models by their own Tanimoto analysis. Halev et al.'s autoencoder still sits at 58.0% for PRRSV and PEDV in System B. Neither method knows which variables encode surveillance. We will nonetheless run both as-is.

Proposed Experiments

The first decisive check is the inflation measurement. On thinned breeding-farm data and on the simulation, we compare each baseline's score on surveillance labels against its score on ground truth, stratified by pathogen. If the gap is under a few AUC points for PRRSV and PEDV, the premise fails, and the project reduces to a benchmark contribution.

The second check is a counterfactual stress test. At test time we shift the testing intensity, for example by doubling finishing-farm testing in the simulation. We then measure how much each model's risk ranking moves. A model predicting infection should barely move, while a model predicting surveillance should move a lot.

The third check is transfer: train on System A and test on System B, and the reverse, reporting feature stability as Tanimoto distance alongside accuracy.

Metrics throughout are:

  • ground-truth AUC and balanced accuracy per tier and per pathogen;
  • sensitivity for finishing farms at a fixed alert budget of flagged farms per week;
  • calibration error per tier;
  • the inflation gap.

Data access is a real constraint. Systems A and B and the Midwest PRRSV cohort are proprietary and reached through Disease BioPortal collaborations. The simulation will therefore be released so that the method comparison can be reproduced without those records.