In computer vision, domain shift typically refers to changes in environment, such as moving from sunny to rainy weather or switching camera manufacturers. In medical imaging, however, domain shift is far more fundamental: it often means moving from a brain to a liver, or from a knee to a chest.
This phenomenon is known as anatomical domain shift. While a deep learning model can easily achieve superhuman precision when segmenting or reconstructing a specific organ, it frequently fails when deployed on a different part of the body. Understanding why these models become trapped in their native anatomical domains—and how to build frameworks that generalize across the entire human body—is a major focus of medical AI research.
Why Models Get Trapped in One Organ
When a neural network is trained on medical images of a specific body region, it does not just learn low-level textures or pixel intensities. Instead, it implicitly bakes the structural, geometric, and physical priors of that specific anatomy into its weights.
For example, McDonagh et al. demonstrated that embedding organ-specific anatomical priors significantly improves the quality of super-resolution models. However, this optimization creates a double-edged sword: the network becomes so dependent on those specific structural shapes that it cannot process different tissue types.
This geometric dependency manifests clearly in spatial alignment tasks. Hou et al. showed that landmark alignment networks optimized for brain structures fail completely when applied to the thorax. Because the network's internal representations are mapped to specific cranial anchor points, it lacks the mathematical flexibility to interpret the entirely different geometric layout of the chest.
How Anatomical Shift Breaks Medical AI Tasks
Anatomical domain shift degrades performance across nearly every major medical imaging workflow:
Image Reconstruction
Models that map sparse sensor data to 3D volumes rely heavily on structural regularizers learned from training data. Lin et al. found that continuous intensity fields optimized for reconstructing sparse-view CBCT scans of the knee fail to resolve images of the chest or head. Because different body parts have vastly different tissue densities and boundaries, Yuan et al. noted that neuroimaging models fail to capture the unique, highly heterogeneous structural properties of hepatic or cardiac tissues.
Generative Modeling and Synthesis
Generative models like GANs and diffusion models suffer from the same localized constraints. Shrivastava et al. demonstrated that diffusion models trained to generate realistic colon tissue cannot adapt to the distinct morphological architectures of breast or lung tissue. Even patch-based generative priors, which operate on smaller image sub-regions to save memory, remain highly specialized to their training anatomy, as observed by Roy et al..
Registration and Motion Modeling
Aligning preoperative scans with intraoperative anatomy requires a deep understanding of how tissues deform. Wu et al. developed highly effective motion correction for liver MRIs, but noted that the framework cannot handle the rapid, cyclical physiological motion profiles of the heart. For rigid or deformable registration, Chen demonstrated that architectures optimized for the lumbar spine lack the inherent flexibility to process the vastly different geometric complexities of the skull or pelvis.
Anatomy vs. Modality: A Surprising Hierarchy
One might assume that changing the imaging technology (e.g., switching from CT to MRI) poses a greater challenge for a model than changing the organ being imaged. However, recent research suggests the opposite.
Van Veen et al. discovered a surprising hierarchy in medical domain adaptation: large language models designed for radiology report summarization generalize far more robustly across different imaging modalities than across different anatomical regions. A model can easily transfer its reasoning capabilities from a chest CT to a chest MRI, but it struggles to transfer knowledge from a chest CT to an abdominal CT. This underscores that anatomy—not modality—is the primary axis of variation in medical data.
Robust Generalization: [Chest CT] ───► [Chest MRI] (Same Anatomy, Different Modality)
Poor Generalization: [Chest CT] ───► [Abdomen CT] (Different Anatomy, Same Modality)
Strategies for Cross-Anatomical Generalization
To move past single-organ silos, researchers are developing frameworks designed to withstand or bypass anatomical domain shift.
Synthetic Data and Contrast Agnosticism
Instead of training networks on real images containing fixed anatomical features, frameworks like SynthSeg (Greve et al.) train networks entirely on randomized synthetic data generated from label maps. While this effectively mitigates contrast and modality variability in neuroimaging, extending this success to highly heterogeneous regions like the abdomen remains an open challenge, requiring the development of entirely new, flexible anatomical priors.
Self-Supervised Anatomical Embeddings
Rather than learning task-specific features, models can be trained to understand the global coordinate system of the human body. Goncharov et al. introduced Anatomical Positional Embeddings (APE), a self-supervised approach that maps voxel-wise coordinates across thousands of unlabeled CT scans. By learning a continuous geometric map of the body, these embeddings allow for zero-shot organ localization and landmark retrieval, providing a universal spatial foundation that spans from the chest to the abdomen.
Multi-Organ Foundation Models
The most direct solution to anatomical shift is scaling the diversity of the training distribution. Initiatives like the Medical Segmentation Decathlon (Simpson et al.) challenge algorithms to perform across 10 diverse organs, forcing the development of general-purpose AutoML solutions.
Similarly, the Ultrasound Representation Foundation Model (URFM) bypasses single-organ limitations by pre-training on over a million images across 15 distinct anatomical regions. By learning high-level semantic features instead of raw pixel noise across a wide variety of tissue structures, these foundation models are beginning to achieve the cross-anatomical robustness required for true clinical deployment.