Unsupervised anomaly detection in brain MRI works by training a generative model only on healthy scans. At test time, the model corrupts a patient scan and repairs it, then subtracts the repaired "pseudo-healthy" scan from the original; whatever the model refuses to rebuild is flagged as lesion. Every such method picks a corruption strength, and every choice fails somewhere. Light corruption lets the model copy the lesion back. Heavy corruption erases the patient's own anatomy along with it. Three further problems sit on top of this. Scans from a different scanner produce residuals the model mistakes for disease. Slice-by-slice models lose continuity along the third axis. And the binary "ground truth" these methods are scored against hides the fact that radiologists disagree about where a lesion ends.
We propose to treat the corruption strength as an axis to measure along instead of a knob to tune. One 3D healthy-anatomy model repairs the volume at several corruption levels. For each voxel we record the fraction of levels at which its residual exceeds a healthy envelope, estimated from a handful of healthy scans from the target scanner. This persistence score runs from 0 to 1. Our central hypothesis is that it tracks the fraction of human raters who would call that voxel lesion: lesion cores persist across many levels, ambiguous borders across few, and scanner-induced residuals across none once the envelope is site-calibrated. If this holds, an unsupervised model trained without a single lesion label would produce calibrated soft segmentations, and it would do so in 3D and under scanner shift.
Core Claim
Let $r_v(t)$ be the absolute residual at voxel $v$ after corruption at level $t$ and repair. Let $q_{s}(v,t)$ be the 95th percentile of the same residual over $N$ healthy scans from target site $s$, registered to a common template. Over $K$ corruption levels $t_1,\dots,t_K$, the persistence score is
$$ p_v = \frac{1}{K}\sum_{k=1}^{K} \mathbf{1}\left[r_v(t_k) > q_{s}(v,t_k)\right], $$
with a 3D connected-component constraint: a voxel counts at level $t_k$ only if its supra-threshold component also exists at a neighboring level.
The claim has three parts, each falsifiable. First, $p_v$ is monotonically related to the rater agreement fraction $a_v$, and is better calibrated against it than raw or severity-averaged residuals. Second, site calibration of the envelope at every level removes most of the scanner-induced false positives documented by Bias in Unsupervised Anomaly Detection in Brain MRI. Third, the multi-level persistence carries information that a single calibrated level does not.
The third part is the one most at risk. We test it directly rather than assume it.
Relation to Prior Work
Several papers motivate the proposal by documenting the gap.
- Bias in Unsupervised Anomaly Detection in Brain MRI trains a 2D variational autoencoder on Siemens ADNI scans. It shows that a Philips shift cuts precision-recall area by 84% under within-subpopulation ("True AD") evaluation, and that the damage goes beyond a mean offset. It proposes no remedy. We supply one and test whether it survives that paper's evaluation protocol.
- IterMask3D is the native-3D answer to the corruption trade-off. It shrinks a mask at test time under edge guidance and stops at a subject-specific threshold. The authors concede that scanner shift still causes false positives, and they score only against binary labels. We share its 3D setting but output a graded map instead of a thresholded mask.
- QUBIQ and Calibrate the inter-observer segmentation uncertainty via diagnosis-first principle establish that rater disagreement is signal, not noise. QUBIQ does so with staged multi-threshold Dice; DiFF does so with diagnosis-weighted label fusion. Both are fully supervised. We ask whether an unsupervised model recovers the disagreement structure for free.
Two papers supply method ingredients.
- Ensembled Cold-Diffusion Restorations for Unsupervised Anomaly Detection (DISYRE v2) is the closest existing work. It restores 2D patches with single-step cold-diffusion at several assumed severities and averages the results. We reuse its single-step restoration to keep $K$ levels affordable. We differ in three ways: we work in 3D, we calibrate each level against target-site healthy residuals, and we use exceedance counts with a stated correspondence to rater agreement rather than an averaged residual.
- Unsupervised Anomaly Detection using Aggregated Normative Diffusion (ANDi) aggregates errors along a diffusion trajectory with a geometric mean and Yen thresholding. It is a second severity-aggregation reference point, but it is 2D and uncalibrated to site.
Two papers constrain the design.
- QUBIQ found that plain deterministic ensembles beat variational probabilistic networks at matching rater distributions. Any graded output we claim must therefore beat a deep ensemble, not a probabilistic U-Net.
- Deep convolutional networks for pancreas segmentation in CT imaging documents organs vanishing between adjacent slices under 2D processing. This is why the persistence components are defined in 3D.
Research Program
The model. We will build a single normative 3D model trained on healthy brain MRI from ADNI, OASIS3 and CamCAN. To fit memory, it will be a patch-based 3D single-step restorer in the cold-diffusion style, trained on synthetic corruptions at a continuous range of severities. At inference it runs once per level. Eight levels keep cost near that of eight forward passes, in the same range as ANDi's reported 34 seconds per full volume.
The envelope. We will estimate the healthy envelope $q_s$ per site from $N \in \{0, 1, 5, 20\}$ healthy target-site scans, where $N=0$ means the source envelope. The curve over $N$ is itself a result, since practicality depends on how few scans suffice.
The analysis. We will study three things. The first is the relationship between $p_v$ and $a_v$: rank correlation, a reliability diagram, and expected calibration error against soft labels. The second is how persistence behaves on pure scanner shift: healthy target-site scans, and the synthetic artifacts (bias field, ghosting, spikes) on which IterMask3D reports AUROC 100.0. The third is cross-slice coherence of the resulting components.
Baselines
Naive reference. The do-nothing reference is the same 3D model at a single, source-tuned corruption level, with raw residuals and no site calibration. This is the configuration whose collapse under scanner shift the Bias paper documents. On ADNI scanner shift, the cited number is the 84% True-AD precision-recall drop for its variational autoencoder. No reported result found on Lacuna for a diffusion or 3D model under that protocol.
Strongest unsupervised 3D competitor. This is IterMask3D, which reports Dice 69.7 on BraTS FLAIR, against 28.5 for a plain diffusion model and 60.0 for a denoising autoencoder. We will run the released method on the identical healthy training cohorts and test splits. We will give it the same target-site healthy scans for threshold tuning, so it is not penalized for lacking calibration data we have.
Strongest severity-aggregation competitor. This is DISYRE v2, which reports average precision 0.79 on BraTS-T2, 0.48 on BraTS-T1 and 0.33 on ATLAS. We will run it in two forms: as published (2D), and with its averaged residual fed through our site-calibrated envelope. The second form separates calibration from persistence. ANDi, which reports precision-recall area 0.422 on Ljubljana, is a secondary reference.
Supervised ceiling for rater-distribution matching. This is a deep ensemble of 3D U-Nets trained directly on QUBIQ's multi-rater labels, following QUBIQ's finding that ensembles win. Top QUBIQ teams report average Dice between 0.80 and 0.85. No reported result found on Lacuna for staged Q-Dice of any unsupervised anomaly detector.
Ablations isolating the key ingredient. The decisive ablation compares three variants built from the same 3D model, the same $N$ target-site scans, and per-voxel envelope z-scoring:
- (a) a single calibrated level, with the level chosen on validation;
- (b) the calibrated mean residual across levels;
- (c) persistence $p_v$.
A further ablation drops the 3D component constraint, and another replaces the 3D model with its 2D slice-wise counterpart.
What counts as winning. Beating the baselines means two things. Persistence must improve staged Q-Dice and calibration error against rater soft labels over (a) and (b) by more than the seed-to-seed spread. It must also recover True-AD precision-recall area under scanner shift to within a stated margin of the in-domain value, where IterMask3D does not.
Why not just…?
Why not just z-score residuals against a few target-site healthy scans at one noise level? This is the most serious threat, and it may explain most of any improvement. Site calibration alone could remove scanner false positives. Ablation (a) is built to settle it. If a single calibrated level matches persistence on both Q-Dice and calibration, then the multi-level component is unnecessary. In that case we would rewrite the contribution as "site-calibrated 3D residuals plus a validated rater-agreement mapping", which is still a gap no paper in the file has closed. The Bias paper's finding that shift degrades discrimination rather than merely offsetting it hints that one level will not suffice, but the experiment decides.
Why not just average restorations across severities, as DISYRE v2 does? Averaging mixes the size of the residual with how often the residual is abnormal. A small, high-contrast spot at one level can dominate the average while being exactly the ambiguous voxel raters split on. Ablation (b) tests whether counting exceedances beats averaging once both are site-calibrated. If (b) matches (c), we report that honestly and adopt the simpler aggregate.
Why not just run a 2D slice model and smooth along the third axis? Smoothing hides slice dropouts but cannot restore a lesion slice the 2D model never flagged, which is the failure the pancreas segmentation work documents. We will run the 2D counterpart with 3D Gaussian smoothing and morphological closing, and measure per-component slice dropout and Q-Dice. If smoothing closes the gap, the 3D model is not worth its memory cost for this task.
Why not just harmonize intensities with histogram matching and heavy augmentation? AMOS still saw drops of over 20% mDice on unseen MRI scanners despite standardized preprocessing, and the Bias paper's shift was not a mean offset. Still, we will include histogram-matched inputs to the naive baseline. That way any gain we attribute to envelope calibration is measured on top of harmonization, not in place of it.
Why not just train a supervised ensemble on multi-rater labels? Where multi-rater labels exist, it will likely win, which is why it is our ceiling. It cannot flag pathologies absent from its training labels, and multi-rater 3D labels are scarce, as QUBIQ notes. The question we answer is how close a label-free model gets.
Proposed Experiments
Experiment 1: agreement correspondence. On QUBIQ's brain tumor and brain growth tasks, which are MRI with multiple raters, we compute $p_v$ and compare it to the rater agreement fraction. Metrics are rank correlation, expected calibration error, and staged Q-Dice over thresholds 0.1 to 0.9. The dimensionality of each task is handled as released, with the 2D variant used where volumes are not provided. The decisive check is a monotone reliability curve with lower calibration error than ablations (a) and (b).
Experiment 2: scanner shift under the honest protocol. We reproduce the True-AD evaluation of the Bias paper with ADNI Siemens, Philips and GE splits, applied to our 3D model and to IterMask3D. We sweep $N$ target-site healthy scans. In parallel, we measure false-positive volume on healthy target-site scans and on synthetic artifacts. The decisive check: persistence keeps healthy-scan false-positive volume near zero while preserving lesion precision-recall area, and the improvement over (a) survives.
Experiment 3: binary-label comparability. On BraTS 2021, ATLAS and ISLES, we threshold $p_v$ at 0.5 and report Dice, average precision and precision-recall area against IterMask3D's 69.7 Dice and DISYRE v2's published average precision. This confirms that graded outputs do not cost conventional accuracy.
The result that would change our mind. If Experiment 1 shows no monotone relationship between persistence and agreement, the rater-agreement framing is wrong. The method would then reduce to a site-calibrated 3D detector, and we would report it as such.
References
- Bias in Unsupervised Anomaly Detection in Brain MRI
- IterMask3D: Unsupervised anomaly detection and segmentation with test-time iterative mask refinement in 3D brain MRI
- QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge
- Calibrate the Inter-Observer Segmentation Uncertainty via Diagnosis-First Principle
- Ensembled Cold-Diffusion Restorations for Unsupervised Anomaly Detection
- Unsupervised Anomaly Detection using Aggregated Normative Diffusion
- Deep convolutional networks for pancreas segmentation in CT imaging