In the world of natural image generation, a "hallucination"—like a cat with five legs—is a humorous glitch. In medical imaging, a hallucination is a clinical catastrophe. When a generative model synthesizes a high-resolution CT scan or a missing MRI sequence, it often prioritizes visual realism (making the image look sharp and "right" to the eye) over anatomical fidelity (ensuring the structures are biologically possible).
Recent research has shifted from unconstrained image synthesis toward "anatomy-aware" frameworks. These models are designed to ensure that every pixel generated adheres to the strict rules of human biology, preventing the creation of non-existent lesions or the erasure of critical diagnostic markers.
The Hallucination Problem
Standard Generative Adversarial Networks (GANs) and Diffusion Models are statistical engines; they learn the distribution of pixel intensities. As noted by Salome Kazeminia and Nassir Navab in GANs for Medical Image Analysis, these models can produce features that appear authentic but are physically impossible.
This lack of structural integrity is a primary barrier to clinical adoption. If a model "cleans up" a noisy scan by smoothing over a tiny, growing tumor, it has failed its medical purpose. Liyue Shen and Lei Xing argue in NeRP that models often bias toward "average" patient characteristics, potentially hallucinating away the subtle, unique pathological changes that clinicians need to see.
Constraining the Latent Space
To solve this, researchers are moving away from "black box" generation and toward constrained architectures. Instead of letting a model dream up an image from scratch, they provide an anatomical "skeleton."
- Structural Priors: Models like BrainMRDiff, developed by Moinak Bhattacharya and colleagues, integrate topology-preserving loss functions. By conditioning the generation on specific anatomical structures and tumor morphology, the model ensures the output is anatomically consistent.
- Multi-Channel Landmarks: In retinal imaging, Zekuan Yu and Yanye Lu demonstrated that providing a model with explicit maps of the vessel tree and optic disc—rather than just the vessels alone—drastically improves the structural accuracy of the synthetic eye.
- Cycle Consistency: Chenyu You and Yi Zhang introduced GAN-CIRCLE, which uses cycle-consistency and structural constraints to recover high-resolution details in CT scans without hallucinating incorrect textures.
From Pixels to Deformation Fields
Perhaps the most intuitive leap in this field is the move from generating intensities to generating deformations.
If you want to synthesize a 3D brain scan of an 80-year-old from a 20-year-old's template, you could try to generate all the pixels from scratch. However, this risks breaking the brain's topology. A more robust approach, proposed by Sophie Starck and Daniel Rueckert in Diff-Def, is to generate a deformation field.
By warping a standard population template to match a specific condition, the model guarantees anatomical integrity. Because the model is "pushing" existing biological structures rather than "painting" new ones, the resulting image is 100% topologically sound. Similarly, the DRDM framework by Yuanhan Mo uses instance deformation to ensure that synthetic data remains plausible for downstream tasks like segmentation.
Task-Aligned Generation
Finally, researchers are realizing that a generative model should "know" what the image is for. Catherine Bouchard and her team developed the Task-Assisted GAN (TA-GAN), which uses biological tasks like segmentation to guide the generation process. If the generated image cannot be accurately segmented by a secondary "expert" model, the generator is penalized.
This "synergy" between understanding and generation—as explored in SynerMedGen by Weiren zhao—forces the model to learn representations that are not just visually pleasing, but functionally and anatomically correct. By enforcing these constraints, the field is moving toward a "Visual Turing Test" where even expert radiologists, as seen in the work on mDDPM, cannot distinguish synthetic scans from reality.