Pathology Randomization for Synthetic Priors and Zero-Shot Detection

Alexandra G. Roberts, Tonmoy Hossain, Mark Olchanyi, Stefan Brandstätter, Jonathan Williams Ramirez, Ana Lawry Aguila, Matthew Rosen, Annabel Sorby-Adams, W. Taylor Kimberly, Juan Eugenio IglesiasarXiv 2026 · arXiv

TL;DR

PathoDraw is a generative framework that enables zero-shot detection, super-resolution, and registration of brain lesions by training neural networks entirely on synthetic data. By "randomizing" pathology—drawing simulated lesions into healthy anatomical maps before rendering them with diverse MRI contrasts—the authors create a single, robust model that generalizes to real-world clinical scans across different diseases and scanner protocols without ever seeing a real patient image during training.

What Problem Does This Paper Address?

Clinical brain MRI archives are a goldmine for medical research, but they are notoriously difficult to use. Scans come from different manufacturers, use varying protocols (like $T_1$-weighted, $T_2$-weighted, or FLAIR), and have vastly different resolutions. Most existing AI models for pathology detection are "contrast-specific," meaning a model trained on $T_1$ scans won't work on FLAIR scans. Furthermore, training these models typically requires massive amounts of manually labeled real-world data, which is expensive and often unavailable for rare conditions.

While synthetic data has previously helped create "contrast-agnostic" models for healthy anatomy, handling pathology remained a major hurdle. Previous synthetic generators either required real lesion masks to start with or were limited to simulating a single specific disease. PathoDraw aims to bridge this gap with a disease-agnostic generative model that handles the "messiness" of clinical data zero-shot.

Methods

The core innovation is Pathology Randomization. Instead of training on real images, the researchers built a sophisticated generator that creates synthetic training examples from scratch:

  1. Anatomy Source: They start with healthy brain label maps (segmentations) from the Human Connectome Project.
  2. Pathology Injection: They "draw" lesions into these healthy maps using three anatomical modes: tract-aligned (following white matter pathways), vascular-aligned (following blood vessel territories), and local (focal lesions like tumors). They even simulate "mass effect," where a lesion physically displaces the surrounding healthy tissue.
  3. Domain Randomization: These augmented label maps are then rendered into synthetic MRI scans. Crucially, the rendering process randomizes everything—contrast, noise, resolution, and artifacts—so the network learns to ignore these variations.
  4. Dual-Head 3D UNet: A single backbone network is trained with two specific tasks:
  5. Anatomy Head: Predicts the underlying healthy labels, effectively "hallucinating" what the brain would look like if the lesion weren't there.
  6. Deviation Head: Estimates the "signed intensity deviation"—the exact difference between the input scan and the predicted healthy anatomy.
Overview of the PathoDraw framework: from synthetic generation to zero-shot inference on real clinical data.

At inference time, the model identifies pathology simply by looking for significant deviations from the predicted healthy state.

Datasets

  • Training: Purely synthetic, using 1,033 healthy label maps from the Human Connectome Project (HCP). No real images were used for training.
  • Evaluation (Real-world data):
    • ATLAS: Stroke lesions on $T_1w$ scans.
  • BraTS: Gliomas across $T_1w$, $T_2w$, and FLAIR.
  • WMH: White matter hyperintensities on FLAIR and $T_1w$.
  • PRES: A hospital-archive cohort of clinical FLAIR scans.
  • ADNI3: Used to test "specificity" (ensuring the model doesn't find lesions in healthy subjects).

Key Insights and Ideas

  • Pathology as a Geometric Prior: The authors treat pathology not as a specific "look" (which changes with MRI contrast) but as a geometric and statistical deviation from healthy anatomy. This allows the model to detect a stroke on $T_1$ (where it's dark) and a tumor on $T_2$ (where it's bright) using the exact same logic.
  • Learning the "Healthy Beneath": By forcing the network to predict healthy labels under the lesion, the model gains a deep understanding of brain structure. This is what enables it to perform registration (aligning a brain to an atlas) while keeping the lesion intact, rather than warping the lesion away as standard tools often do.
  • The Power of Signed Deviation: Predicting a continuous "deviation map" rather than a binary mask provides much more information. It captures the "polarity" of the lesion (brighter or darker than tissue) which is essential for multi-contrast robustness.
Figure 2. Pathology randomization and the network, one draw of the generative model. The top row shows the healthy labels, the drawn mode and its eligible region, the random field whose excursion set is the lesion, the injected label map (pink) and the rendering (Tab. S1, Appendix A). Below, the rendering enters the contrast-agnostic 3D UNet backbone, whose anatomy head predicts the healthy labels and deviation head the signed deviation. The same draw passes through the task networks, super-resolved from 6 mm slices and registered to another draw’s healthy rendering, with the warped lesion (pink), target brain (cyan) and Jacobian determinant.

Key Results

  • Zero-Shot Detection: PathoDraw matched or exceeded the performance of state-of-the-art anomaly detectors that were specifically trained on real data or restricted to single contrasts. For example, it achieved a "ceiling Dice" score of 0.657 on BraTS $T_2w$ and 0.369 on the ATLAS stroke dataset.
  • Super-Resolution: The model successfully restored fine details in lesions from thick-slice (3mm-6mm) clinical scans, outperforming standard interpolation methods and healthy-only priors.
  • Registration: In tasks requiring aligning pathological brains to the MNI152 atlas, PathoDraw preserved lesion geometry significantly better than the popular SynthMorph tool, which tends to distort or "erase" lesions during the warping process.
Figure 3. Exemplar cases. One model on test scans from four diseases, three contrasts and 37 sites; one column per dataset (ATLAS T1w, BraTS T1w/FLAIR/T2w, WMH FLAIR/T1w, clinical posterior reversible encephalopathy syndrome (PRES) FLAIR), with the input image, ground truth (green) and our segmentation (pink), and the signed deviation map, at protocols from 1 mm isotropic to 6 mm slices.

Limitations and Future Work

  • Manifold Enclosure: The model can only detect lesions that fall within the "shape manifold" of its generator. If a real-world pathology has a shape or placement completely unlike the randomized synthetic ones, the model might miss it.
  • Anatomical Modes: While the three current modes (tract, vascular, local) cover a lot of ground, adding more specific modes (e.g., for multiple sclerosis or specific hemorrhages) could further improve performance.
  • Healthy Rendering Gaps: The authors noted that some false positives occur because the synthetic "healthy" rendering doesn't perfectly match the complex intensity distributions of real healthy tissue in all scanner types. Refining the underlying healthy generator remains an area for improvement.

Summary updated

Abstract

Hospital brain imaging archives hold a diverse array of pathology at sample sizes beyond prospective studies, particularly for underrepresented populations and diseases. To unlock uncurated clinical data and scale analysis, methods are needed to detect abnormalities across contrasts and resolutions. Synthetic data has provided robust, contrast-agnostic models for a variety of human neuroimaging tasks in healthy anatomy, but pathology detection and other tasks in clinical settings still require real patient data, contrast-specific models, or disease-specific generators. We propose PathoDraw, a generative model of brain MRI with pathology. A conditional distribution of lesions given the healthy anatomy is drawn into the label map before contrast-agnostic rendering, and a single network estimates healthy anatomy with exact signed intensity deviation of any input scan. The contrast-agnostic 3D UNet learns the healthy labels under the lesion and the signed deviation directly and identifies pathology via robust tissue statistics and learned features, despite no training on real pathology. We show zero-shot detection on public stroke, tumor, and white matter hyperintensity datasets as well as archived clinical data, spanning $T_1$w, FLAIR and $T_2$w at various resolutions under a recent unsupervised anomaly detection benchmark protocol. Our single model exceeds or performs comparably to contrast-specific anomaly detectors trained on real scans and keeps healthy anatomy intact. The same generator trains networks for super-resolution and for registration in the presence of pathology. Our model, robust across acquisition and disease, enables the use of uncurated, archived data for clinical analysis. Our code is available at https://github.com/agr78/Pathology-Randomizer.
BibTeX
@misc{roberts2026pathology,
  author = {Alexandra G. Roberts and Tonmoy Hossain and Mark Olchanyi and Stefan Brandstätter and Jonathan Williams Ramirez and Ana Lawry Aguila and Matthew Rosen and Annabel Sorby-Adams and W. Taylor Kimberly and Juan Eugenio Iglesias},
  title = {{Pathology Randomization for Synthetic Priors and Zero-Shot Detection}},
  year = {2026},
  eprint = {2611.21975},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21975},
}
Figure