Generative models have reached a point where they can synthesize medical images that are virtually indistinguishable from real scans to the human eye. However, in clinical contexts, "looking real" is not the same as "being right." A synthetic brain MRI might possess perfect photographic texture while containing anatomically impossible structures or missing critical pathological markers. This discrepancy has led to a new research direction: developing metrics that prioritize anatomical fidelity and clinical validity over mere visual realism.
The Realism Gap
Traditional generative evaluation relies on metrics like Fréchet Inception Distance (FID) or Peak Signal-to-Noise Ratio (PSNR). As Konz et al. and Müller‐Franzes et al. argue, these metrics are optimized for natural image distributions (like ImageNet) and primarily measure visual texture and statistical similarity.
In medical imaging, this creates a "realism gap." Jafrasteh et al. demonstrate that models can achieve high visual fidelity scores while producing "anatomical hallucinations"—subtle morphometric errors that would mislead a clinician. Similarly, the authors of AG-LDM note that FID often fails to identify structural errors that are biologically impossible, such as incorrect organ volumes or displaced tissues.
Beyond Statistical Distribution
To bridge this gap, researchers are moving toward evaluation frameworks that incorporate domain-specific knowledge. Instead of treating an image as a collection of pixels, these metrics treat it as a biological map.
- Morphometric Accuracy: Jafrasteh et al. introduced WASABI (Wasserstein-Based Anatomical Brain Index), which uses deep-learning segmentation to ensure synthetic brains are morphometrically plausible.
- Structural Consistency: Volokitin et al. proposed the Realistic Atlas Score (RAS), which uses image registration to check if generated volumes align with known anatomical templates. They caution, however, that registration scores can be misleadingly high for blurry images, necessitating a balance between alignment and image diversity.
- Biological Markers: In pathology, Quiros et al. emphasize that metrics must capture indicators like cell density and tissue architecture. Dubey et al. validated their virtual staining model using automated cell counting, ensuring that biological markers were not just visually present but correctly localized.
Task-Specific and Downstream Validation
A recurring theme across recent work is that the ultimate test of a synthetic image is its downstream utility. If a diagnostic AI trained on synthetic data performs well on real patients, the synthetic data is likely valid.
Hamamci et al. demonstrated this by showing that models trained on synthetic 3D chest CTs could outperform those trained on limited real datasets. This "turing test" for utility is often more telling than statistical scores. For human-centric generation, Zhang et al. utilized pose estimators to measure anatomical correctness, arguing that task-specific tools provide a more reliable indicator of quality than generic classification scores.
Addressing Structural Glitches
The need for these metrics is underscored by the "peculiar images" often produced by diffusion models, such as humans with extra limbs. Youn et al. attribute these to the sampling process getting stuck in local minima and suggest that automated evaluation of logical consistency remains a major open challenge.
In the medical domain, these glitches are not just visual artifacts but clinical risks. As Friedrich et al. and the authors of Pflügers 2024 conclude, the field is shifting toward a standardized requirement: synthetic medical data must be validated against the precise anatomical and pathological features required for reliable diagnosis, ensuring that AI-generated scans are medically "true" rather than just "pretty."
Key Foundational Work
Modelling the Distribution of 3D Brain MRI Using a 2D Slice VAE (2020) — contributed 2 concept elements to this direction.
Improving statistical power of glaucoma clinical trials using an ensemble of cyclical generative adversarial networks (2021) — contributed 1 concept elements to this direction.
PathologyGAN: Learning deep representations of cancer tissue (2020) — contributed 1 concept elements to this direction.