Generative models can now turn an emergency CT perfusion scan into a synthetic diffusion MRI. CT perfusion is a contrast-enhanced CT series that shows blood flow in the brain. Diffusion MRI is the scan neurologists trust most for seeing dead tissue, and in the emergency room it is often unavailable. The danger is that these models "hallucinate." They paint a plausible infarct where none exists, or erase a small real one. In acute stroke that matters because a treatment decision hangs on a volume. How large is the irreversibly damaged core, and how much at-risk tissue surrounds it? Is the patient still inside the time window where clearing the clot helps more than it harms? The field currently scores these models with pixel error and perceptual similarity. A model can do well on both and still push patients across a treatment threshold.
We propose to judge any lesion-producing model, generative or not, by one question: does it flip treatment eligibility more often than two expert neuroradiologists disagree with each other? That rate of disagreement between experts, measured on the same cases, sets the noise floor. A hallucination only counts clinically if it pushes the model's decision-flip rate above that floor. Three things follow from this. First, the evaluation is tied to the decision, not to pixels. Second, the model is trained against the spread of several expert annotations rather than a single consensus mask. Third, a staged prospective "silent-mode" deployment tests whether the offline noise-floor comparison predicts behaviour in a live stroke pathway.
Core Claim
Our claim is that hallucination in stroke imaging should be defined relative to the therapeutic decision and to inter-observer variability, not relative to a single ground-truth mask.
Concretely, fix a decision rule $D_\tau$ that maps a lesion mask to "treat / do not treat." Here $\tau$ is the set of core-volume and mismatch thresholds, which depend on time since onset. For each case $i$, let $m$ be the model's mask and $r, r'$ two distinct expert masks. The quantity we care about is the excess flip rate:
$$\Delta_\tau = \Pr_{i,r}\big[D_\tau(m_i) \neq D_\tau(r_i)\big] - \Pr_{i,r\neq r'}\big[D_\tau(r_i) \neq D_\tau(r'_i)\big].$$
The first term is how often the model disagrees with a randomly chosen expert. The second is how often two experts disagree with each other. A model is decision-safe at $\tau$ if $\Delta_\tau$ is non-inferior to zero within a pre-registered margin. Fabricated and erased lesion components are tallied separately, weighted by whether they changed $D_\tau$.
We expect three results. Current synthesis models trained with a single-expert lesion loss will show positive $\Delta_\tau$ concentrated on cases near the threshold, even when their mean absolute error is competitive. Training against rater distributions will shrink $\Delta_\tau$ more than it shrinks pixel error. And the offline $\Delta_\tau$ will rank models the same way a prospective shadow deployment does, while Dice and FID will not.
Why This Intersection, and What Prior Work Leaves Open
The closest work is cLDM-PT. It post-trains a latent diffusion model to synthesize diffusion MRI (DWI) and apparent-diffusion-coefficient (ADC) maps from CT perfusion in 817 stroke patients. It adds an image-space loss and an expert-mask-weighted lesion loss, and reports DWI MAE 0.072, ADC MAE 0.052, and a 12.4% lesion-MAE reduction over a base latent diffusion model. We reuse its architecture as the main generative baseline. We depart from it in three ways: its lesion loss uses one expert mask per case, it reports FID alongside MAE, and it never asks whether errors change treatment.
Two negative results motivate that departure. Rethinking Perceptual Metrics for Medical Image Translation showed that FID has near-zero correlation with downstream bone-segmentation Dice in spine MRI-to-CT translation. Huang et al. showed that at high undersampling, adversarial objectives trade data consistency for sharpness.
The noise floor itself comes from inter-observer work. Observer Agreement on CT Perfusion measured agreement between clinicians on perfusion maps in 24 trial cases. Agreement was Krippendorff's α of 0.62 for cerebral blood flow and 0.81 for penumbra maps, and intra-observer α ranged from 0.29 to 0.80. ISLES'24 reports inter-rater Dice of 0.86 to 0.90 on follow-up infarcts. Both results say "ground truth" is itself noisy. Neither says how much of that noise crosses a treatment threshold, which is what we will measure.
For modeling the rater spread we borrow method rather than motivation. Pionono represents each annotator as a Gaussian in an 8-dimensional latent space: the mean captures that rater's systematic bias and the variance captures their inconsistency. The Latent Doctor Model matches panel entropy (Spearman up to 0.728 on 20-pathologist ISUP grading). Both were evaluated only on offline histopathology. DiFF weights rater masks by how much they help a downstream classifier. Its authors warn that this couples the fused label to the classifier architecture. For that reason we do not learn a fused label from the treatment decision. We use the decision only for evaluation and for a boundary-focused loss term, which avoids that coupling.
The therapeutic-window literature supplies the time dimension. Zheng et al. and Wu et al. show that post-stroke tissue biology changes sign across acute and subacute phases; in Wu et al., microglial STING activity peaks at 7 days. This matters for us in one practical way. ISLES'24 follow-up MRI is acquired anywhere from 2 to 9 days after stroke. The "final infarct" label therefore partly reflects when the scan happened, and we condition on and stratify by follow-up day. DeepISLES and the Mia prospective mammography rollout supply the deployment template: external validation first, then staged live implementation with human arbitration.
Research Program
The first component is a multi-rater annotation layer. On the ISLES'24 training and test cases, which include acute CT perfusion, follow-up DWI/ADC, onset-to-imaging times, and 3-month modified Rankin Scale outcomes, five neuroradiologists will independently contour follow-up infarcts. A 20% subset will be re-read after a washout period to estimate intra-rater consistency. Where a collaborating centre can supply CT perfusion paired with near-simultaneous acute DWI, as in the cohort used by cLDM-PT, the same protocol is applied to the acute core. That gives us both the synthesis task and the tissue-fate prediction task.
The second component is the evaluation protocol itself. We sweep $\tau$ across a range of core-volume and mismatch thresholds, stratified by onset-to-imaging time. We treat thresholds as parameters because guidelines differ between sites. For each model we compute $\Delta_\tau$ with bootstrap intervals over both cases and rater pairs. We also decompose flips into fabricated components (model lesion with no overlap in any rater mask) and erased components (a lesion present in the majority of raters but absent from the model). Small scattered lesions get their own tally, since ISLES 2022 reports up to 126 disconnected lesions in a single patient.
The third component is rater-distribution training. The cLDM-PT backbone and an nnU-Net-style direct segmenter are each trained in two ways. The first uses a single consensus lesion loss. The second samples a rater latent in the style of Pionono at each step and applies the lesion loss against that rater's mask. It also adds a small term that penalizes the gap between predicted lesion volume and the rater-sampled volume, and that gap is weighted by how close the case sits to $\tau$. At inference we draw several samples, report the fraction that land on each side of the threshold, and defer to a human when that fraction is ambiguous.
The fourth component is prospective shadow deployment. At one stroke centre the best model runs silently on every emergency CT perfusion scan for several months, and its decision is logged against the treating team's decision and the follow-up MRI. A second phase adds a reader study of the kind in the COVID-19 CT study: neuroradiologists read with and without the model's output and its sample spread, separated by a washout period. We record decision time, inter-reader κ, and flip rate.
Baselines
The naive reference is the clinical status quo: thresholded perfusion maps from the standard vendor software, read by the treating team. This has no learned component, and its $\Delta_\tau$ against our follow-up panel is the figure any model must at least match. Everything is scored on identical ISLES'24 cases and identical rater panels.
The strongest discriminative competitor is DeepISLES DeepISLES. It is an ensemble of three segmentation networks and reports median Dice 0.82 and F1 0.86 on 1,685 external scans. That result is on DWI segmentation, not on CT perfusion input, so we retrain its components on CT perfusion to follow-up infarct under the same data splits, the same GPU budget, and the same hyperparameter-search budget as our models. No reported result was found on Lacuna for any algorithm on the ISLES'24 tissue-fate benchmark, and none for decision-flip rates on any stroke dataset. The strongest generative competitor is cLDM-PT reimplemented under its published losses, reference MAE DWI 0.072 and ADC 0.052 on its own cohort. We compare to those numbers only as a sanity check of the reimplementation where paired acute data are available. Beating either competitor means a lower $\Delta_\tau$ with non-overlapping bootstrap intervals at clinically used thresholds, with no worse Dice.
The key ablation holds the architecture fixed and swaps only the training target: single consensus mask versus rater-sampled masks. It isolates whether modeling the rater spread, rather than extra capacity or sampling, is what closes the gap to the noise floor.
Why not just…?
Why not just segment CT perfusion directly and skip synthesis? This is the most serious threat, and the proposal is built so it can win. The evaluation protocol applies equally to a direct segmenter, and the retrained DeepISLES baseline is exactly this alternative. If it matches synthesis on $\Delta_\tau$, then the synthetic image adds only something for a reader to look at. The reader-study phase then decides whether that is worth the extra hallucination surface. The motivation for synthesis rests on the claim, stated by cLDM-PT, that early ischemia is less visible on CT. Human agreement on raw perfusion maps is also weak (cerebral blood flow α 0.62). Neither fact proves that a discriminative model cannot recover that signal.
Why not use deep ensembles or dropout for uncertainty instead of a rater latent? Ensembles capture model uncertainty, not the systematic disagreement between annotators. The Latent Doctor Model found ensembles competitive but costly as the rater panel grows. Pionono reports that pixel-wise uncertainty baselines produce noise rather than coherent alternative contours. Still, a five-member ensemble trained on consensus masks is included as an additional arm. If its sample spread straddles thresholds as well as rater-sampled training does, we will report that the rater latent is unnecessary for this task.
Why not just train on a majority-vote mask and report Dice against it? Majority vote hides exactly the cases that matter, those near the threshold where experts split. DiFF found majority fusion suboptimal on ambiguous boundaries. Our ablation directly measures whether consensus-trained models have positive $\Delta_\tau$ concentrated near $\tau$ despite good Dice.
Why not rely on large retrospective multicentre validation instead of prospective deployment? The Mia rollout showed that viability depended on arbitration rates (7–11% extra reads) that no offline set measures. Horng et al. stressed that retrospective AUC does not guarantee a change in clinician behaviour. The shadow phase is kept deliberately cheap, since outputs are logged but not shown, precisely to test whether offline $\Delta_\tau$ is enough on its own.
Decisive Checks
Four results would decide the proposal. The first is whether, across all models, offline $\Delta_\tau$ rank-correlates with the prospective shadow-mode flip rate better than Dice, MAE, or FID do. If it does not, the metric fails its purpose. The second is whether rater-sampled training lowers $\Delta_\tau$ relative to consensus training at matched Dice. If it does not, the training ingredient is unnecessary. The third is whether fabricated or erased components account for most of the excess flips, and whether this differs by onset-to-imaging window and follow-up day. The fourth, from the reader study, is whether showing the sample spread raises inter-reader κ without lengthening decision time. The COVID-19 CT study saw κ rise from 0.34 to 0.58 with a discriminative aid, and we test whether a generative aid with visible uncertainty repeats or reverses that effect.
References
- Lesion-Aware Post-training of Latent Diffusion Models for Synthesizing Diffusion MRI from CT Perfusion
- Rethinking Perceptual Metrics for Medical Image Translation
- Fast MRI Reconstruction: How Powerful Transformers Are?
- Observer Agreement on Computed Tomography Perfusion Imaging in Acute Ischemic Stroke
- ISLES'24 -- A Real-World Longitudinal Multimodal Stroke Dataset
- Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
- The Latent Doctor Model for Modeling Inter-Observer Variability
- Calibrate the Inter-Observer Segmentation Uncertainty via Diagnosis-First Principle
- Cdh5-mediated Fpn1 deletion exerts neuroprotective effects during the acute phase and inhibitory effects during the recovery phase of ischemic stroke
- STING inhibition suppresses microglia-mediated synapses engulfment and alleviates motor functional deficits after stroke
- DeepISLES: a clinically validated ischemic stroke segmentation model from the ISLES'22 challenge
- Prospective implementation of AI-assisted screen reading to improve early detection of breast cancer
- AI support for accurate and fast radiological diagnosis of COVID-19: an international multicenter, multivendor CT study
- Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning