Letting Normative Detectors Forget the Scanner but Not the Disease

A normative model learns what healthy anatomy or physiology looks like and flags anything that departs from it. That lets it catch rare diseases it never saw labeled examples of. The same design has a weakness in hospitals that run for years. When a scanner is upgraded or a data pipeline changes, healthy patients start to look "unusual" to a model frozen on old data, and the false-alarm rate climbs. A benchmark on ten years of network traffic made this concrete. Normal data drifted faster than anomalies, and a detector that scored 91.50 area under the ROC curve (AUROC) on in-period data fell to 34.96 on data from several years later, worse than random (AnoShift). In a clinical setting that failure turns directly into alert fatigue and wasted arbitration reads. Clinical deployment work shows that this extra workload is what decides whether a tool survives in practice (Ng et al.).

The usual clinical fix for drift is to retrain on fresh labels, and labels arrive months late (Guo et al.; Ötles et al.). A normative model has an unusual advantage here: most incoming exams are normal, so it can in principle update itself from unlabeled data. The obvious way to do that is to refit on recent exams, and it carries an obvious danger. If disease becomes more common, a model refit on recent data learns that disease is normal. This proposal resolves the tension with a narrow rule. The model may absorb only the drift that the hospital's own acquisition records can explain, such as scanner model, software version and protocol. It does this through a small set of parameters keyed to those records and refit on unlabeled data after the likely-abnormal cases are removed. Drift that no acquisition change explains is left for the detector to flag.

Core Claim

For a normative detector under temporal drift, the main danger is that its picture of "normal" goes stale. A stale picture of normal damages the detector's ranking of cases, which a threshold change cannot repair. It can nonetheless be repaired without outcome labels, provided adaptation is restricted to acquisition-explained directions.

We make three testable predictions on longitudinal imaging data:

  1. Re-thresholding the detector to hold a fixed alert rate fixes the alarm count but not the AUROC loss.
  2. Refitting the whole model on recent unlabeled data restores AUROC after a scanner change but loses it when the share of abnormal exams rises.
  3. Acquisition-keyed adaptation recovers most of the scanner-induced loss while staying sensitive to a rise in abnormal prevalence.

The asymmetry mirrors the supervised clinical finding that recalibration restores calibration but not discrimination (systematic review). Unlike the supervised case, the normative case permits a label-free repair of discrimination, and that is the claim this proposal tests.

Relation to Prior Work

Three papers motivate the gap.

  • Inlier drift. AnoShift documented the inlier-drift failure. It also showed that a student model distilled from the old detector while training on unlabeled new data beats naive fine-tuning by about 3% AUROC. That work used tabular network logs with no notion of pathology prevalence, so it never faced the risk of absorbing disease.
  • Scanner upgrades. Bordner et al. showed that a sacroiliac MRI model degraded on 5- and 10-year follow-ups because scanners and reconstruction were upgraded.
  • Normative brain models. Lopez Pinaya et al. built a normative adversarial autoencoder on 11,034 UK Biobank controls conditioned on age and sex. Its deviation scores reached 0.84 to 0.98 AUROC for Alzheimer's versus controls across five external cohorts. That model treats acquisition as invariant, and our method extends its conditioning idea from demographics to acquisition.

Two papers constrain the design.

  • Invariance discards signal. Guo et al. found on more than 50,000 MIMIC-IV admissions that invariance-seeking domain generalization and unsupervised domain adaptation never beat plain empirical risk minimization. The reason was that drifting features carried clinical signal. This rules out a design that forces representations to ignore time wholesale.
  • Style invariance is untested on real drift. ADShift does exactly that kind of global style-invariance training plus test-time feature matching. It reports gains of more than 10% AUROC, but only on synthetic corruptions and photo-to-sketch transfer.

Our method sits between these two. It does not ask the representation to be invariant in general. It lets a few parameters move, and only along axes the hospital's records name as technical.

Three papers supply the method.

  • Contamination estimation. We estimate the fraction of abnormal cases in each unlabeled window with γGMM, which models the contamination fraction as a Bayesian posterior and is designed for fractions up to 25%.
  • Mixed unlabeled data. DDAD showed that unlabeled, mixed chest X-rays can be exploited rather than discarded.
  • Evaluation protocol. We evaluate with the rolling train-on-past, test-on-future protocol of EMDOT, which showed that random splits overstate deployed performance.

Research Program

Detector form. The detector reconstructs or embeds an exam $x$ conditioned on demographics $c$ and an acquisition key $a$. The key $a$ is read from image headers or pipeline logs: manufacturer, model, software version and protocol family. It scores $s(x) = d\big(x, \hat{x}(x; \theta, \phi_a, c)\big)$, where:

  • $d$ is a perceptual reconstruction distance in the style of Bercea et al.;
  • $\theta$ are shared weights frozen after training on the earliest normal cohort;
  • $\phi_a$ are per-acquisition scale-and-shift parameters inserted into normalization layers.

These parameters are a small fraction of the model, so they can absorb intensity and texture changes, but they lack the capacity to learn new lesion shapes.

Adaptation rule. When the stream reveals a new key $a'$, or after each monthly window for existing keys, we do three things:

  1. Score the window's exams.
  2. Estimate the abnormal fraction $\hat\gamma$ with γGMM.
  3. Discard the top $\hat\gamma$ plus a safety margin of scores, and refit only $\phi_{a'}$ on the remaining exams, using the frozen detector's own loss.

Exams whose acquisition metadata is missing or inconsistent fall back to the nearest known key and are logged. Silent pipeline faults were real in practice: one live clinical pipeline failed on 10 of 356 days (Ötles et al.). We therefore report results separately for the fallback cases.

Evaluation setting. The prospective-validation direction enters as the evaluation lens rather than as a method. Every number is computed on chronological backtests where only past data is used. Beyond AUROC we report one operational quantity: arbitration workload at matched sensitivity, measured as extra human reads per 1,000 exams. This follows the additional-reader design of Ng et al., where the detector screens exams already read as normal.

Data. The primary imaging testbed is MIMIC-CXR, which spans 2009 to 2020 and was used for temporal analysis by EMDOT.

  • Year groups come from the dataset's anchor-year groupings.
  • Acquisition keys come from image headers where recorded.
  • Normal and abnormal reference labels come from report-derived findings. These are used only for evaluation and for selecting the first-era normal training set.

The second testbed is structural brain MRI from ADNI, whose phases include documented changes in scanner and field strength. On ADNI we rebuild the Lopez Pinaya normative model and add an acquisition key, so the scanner question can be asked of a published normative pipeline. Kyoto-2006+ from AnoShift serves as a sanity benchmark. It has no acquisition metadata, so it tests the unconditional and threshold arms against the published numbers.

Baselines

Frozen detector (do-nothing reference). This is the same detector trained once on the earliest normal era and never touched, run on identical windows. Every method must beat it on later-era AUROC to justify any maintenance at all. AnoShift's LOF collapse from 91.50 to 34.96 AUROC shows how far this reference can fall.

Fixed-alert-rate re-thresholding. This is the frozen detector with its threshold reset each window so the flag rate matches the first era's rate. It is the cheapest label-free fix and directly targets alarm counts. To beat it, our method must improve AUROC and lower workload at matched sensitivity, not merely match the alarm count.

Unconditional rolling refit by distillation. This is AnoShift's teacher-student distillation on each new unlabeled window. It is the strongest existing label-free method for drifting normative detectors, with about +3% AUROC reported over standard training on Kyoto-2006+. We give it the same contamination trimming, the same windows and the same tuning budget, and we let it update all parameters. On medical imaging under temporal drift, no reported result was found on Lacuna for it or for any unsupervised detector on MIMIC-CXR across years. Our success criterion is that the two methods are statistically indistinguishable after scanner changes, while our method is clearly better in windows where abnormal prevalence rises.

Global style invariance. This is GNL from ADShift, trained and run with the released code (mala-lab/ADShift) on the same backbone family and normal data. Its reported figure is a gain of more than 10% AUROC on synthetic shifts. We want to know whether that gain survives real multi-year drift; if it does not, that would extend Guo et al.'s supervised negative result to normative models.

Ablations isolating the key ingredient. Two ablations separate the contributions of our design.

  • Calendar-keyed adaptation uses the same small parameter set, but keyed by calendar month instead of acquisition metadata. It separates the value of the metadata from the value of low-capacity adaptation.
  • Full-model adaptation keyed by acquisition separates the value of restricting capacity.

A third ablation removes contamination trimming, to measure how much γGMM matters.

Why not just…?

Why not just reset the threshold to a fixed alert rate? That keeps the alarm count steady but cannot reorder cases. A detector whose AUROC has fallen below 0.5, as in AnoShift's distant-future split, ranks normals above abnormals, and no threshold fixes that. Holding a fixed alert rate also silently hides a real rise in abnormal prevalence. This is the supervised recalibration story transplanted, since recalibration restores calibration but not discrimination (systematic review). If our backtests show that scanner drift in MIMIC-CXR mostly shifts scores without reordering them, this baseline wins, and we will say so. The scanner-change windows settle the question directly by comparing AUROC, which thresholds cannot change, against workload.

Why not just refit the whole detector on recent unlabeled exams? This is the real threat. It is what AnoShift's distillation does, and on pure technical drift it may match or beat our restricted adaptation, because it has more capacity. Its weakness is predicted rather than yet observed: when abnormal findings become common, refitting teaches the model that they are normal. Trimming with γGMM limits but does not remove the effect, and the estimator assumes contamination of at most 25%. The prevalence-injection experiment below decides the matter. If unconditional refit holds AUROC there as well, the acquisition key adds nothing and the proposal reduces to applying AnoShift to clinical data.

Why not just make the detector invariant to style, as ADShift does? Guo et al. found that invariance constraints hurt clinical models because drifting features carried signal. Global style invariance in a chest X-ray could likewise erase density and texture cues that indicate disease. ADShift's evidence is limited to synthetic corruptions and domain transfer. We run it as a baseline rather than argue the point.

Why not just use a supervised classifier with periodic retraining? Retraining waits for labels that arrive with long delays. A supervised model also cannot flag pathologies absent from its label set, which is the reason normative modeling is used in the first place (Lopez Pinaya et al.). We report a label-delayed supervised classifier as context, but it answers a different question.

Decisive Experiments

Scanner-change test. On MIMIC-CXR and ADNI we locate windows where a new acquisition key appears while report-derived abnormal prevalence is roughly stable. We measure AUROC and workload at matched sensitivity over the first one, three and six months after the change. The prediction is that our method and unconditional refit both recover most of the frozen detector's loss within a few months, while fixed-rate re-thresholding recovers workload but not AUROC.

Prevalence-injection test. In later windows with no acquisition change, we resample the stream so that the abnormal fraction rises from its natural level to two or three times that level, staying within γGMM's 25% range. The prediction is that unconditional refit loses AUROC as it absorbs abnormal cases, while acquisition-keyed adaptation does not, because no new key triggers adaptation and the per-key parameters lack the capacity to learn lesions. A crossed version applies both a scanner change and a prevalence rise together, and is the single result that most directly supports or refutes the core claim.

Silent-fault test. We corrupt or remove acquisition metadata on a fraction of exams, mimicking pipeline faults. This measures how gracefully the fallback degrades and whether metadata errors cause the method to absorb or miss drift.

Falsification criteria. The proposal is falsified in two cases. The first is if re-thresholding matches it on workload after scanner changes. The second is if unconditional refit matches it under prevalence injection.