A physical phantom is a manufactured stand-in for a patient: a 3D-printed or cast object whose internal "organs" have shapes the builder knows exactly. Phantoms are poor copies of real biology. They have no breathing, no blood flow, no bowel gas, and no layered tissue. For that reason, models validated only on phantoms routinely fail on patients. Phantoms do have one property that patient data almost never has, though. The same object can be scanned in CT and in MRI, and the two scans then show identical geometry with a known voxel-to-voxel correspondence and a free, exact label map. This proposal treats phantoms as paired dictionaries between imaging physics, not as realistic training images. They would teach a segmentation network how "the same material" looks in CT versus MRI. Unlabeled patient scans would teach it what real anatomy looks like.
The opportunity is concrete. The M3DA benchmark shows that when a network trained on labeled abdominal MRI is applied to CT with no target labels, Dice (overlap between predicted and true masks, from 0 to 1) is 0.032. The best unsupervised adaptation method, entropy minimization with heavy augmentation, reaches only 0.388 (M3DA). Every method in that benchmark has to infer the MRI-to-CT correspondence from unpaired patient images. We ask whether a handful of paired phantom scans can supply that correspondence directly. We also ask how many expert-annotated patient scans such pairs are worth.
Core Claim
The phantom-to-patient gap and the CT-to-MRI gap are different axes, and phantoms are informative about only one of them. Phantom materials can be tuned to approximate the contrast of each tissue class in both CT and MRI, so co-registered phantom pairs carry real signal about the physics shift. They carry no reliable signal about biological shape and texture, which is where the phantom-limitations literature documents failure: missing tissue layers in palpation (Autonomous Robotic Tissue Palpation), missing specularities and fluids in colonoscopy (Rendering with a Gut Feeling), and missing respiratory dynamics (Fusing Prediction and Perception).
We therefore claim the following. A network that receives labeled patient scans in the source modality, unlabeled patient scans in the target modality, and a correspondence loss on paired phantom scans will clearly exceed the 0.388 adaptation ceiling on MRI-to-CT abdominal segmentation. The gain will come from the pairing itself, not from the phantoms serving as extra labeled images. The gain will also shrink predictably wherever phantom contrast deviates from patient contrast.
This differs from all prior work in the file. Meta-hallucinator learns synthetic intermediate representations between MRI and CT but still needs one to four annotated target-modality patient scans, and it never uses correspondence. MiDSS builds bridge domains by Fourier amplitude mixing and copy-paste, but only for shifts within one modality. The CT dose phantom study (Impact of CT dose on AI performance) found that phantom findings transferred to patient CT, but it stayed within one modality and never used phantom masks for adaptation. The fiducial-marker work (NOVA-FM) uses phantoms for MRI/CT fusion, but for registration hardware, not learning.
Research Program
Phantom data. We would build a small family of about five abdominal phantoms rather than one. The number matters because a single geometry invites memorization. Each phantom would have liver, spleen, and kidney compartments, printed from patient-derived shapes and filled with materials chosen to be visible in both CT and MRI. Each phantom would be scanned on several CT scanners at several doses and on several MRI scanners and sequences. This mirrors the multi-scanner design of CT4Harmonization, which covered 13 scanners and 5 doses (Impact of CT dose on AI performance). Labels come from the print design. Cross-modality alignment is rigid, because the object does not deform, so it is far more accurate than any patient CT-MRI registration. A cheaper parallel arm would use computational phantoms simulated in both modalities, which lets us test whether physical manufacture is necessary at all.
Training. The backbone is a standard nnU-Net, matching M3DA so the numbers are comparable. It trains on three streams. The first is supervised segmentation on labeled patient MRI. The second is entropy minimization on unlabeled patient CT, which is the strongest M3DA recipe. The third is a paired-correspondence loss on phantoms:
$$ \mathcal{L}_{\text{pair}} = \frac{1}{|V|}\sum_{v \in V} \left\| g\big(f(x^{\text{CT}})_v\big) - g\big(f(x^{\text{MR}})_v\big) \right\|^2 + \mathcal{L}_{\text{seg}}(x^{\text{CT}}, y) + \mathcal{L}_{\text{seg}}(x^{\text{MR}}, y). $$
The symbols are defined as follows. $x^{\text{CT}}$ and $x^{\text{MR}}$ are a registered phantom pair with shared label map $y$. $f$ is the encoder. $g$ is a small projection head. $V$ is the set of phantom voxels, and $v$ is one voxel. The first term forces the same physical point to produce the same features regardless of imaging physics. The segmentation terms keep those features semantically meaningful.
Guarding against shortcuts. To limit transfer of phantom artifacts, the correspondence loss is applied only to encoder features. The decoder sees phantoms only through the two segmentation terms, which are down-weighted. We will also train a probe to classify phantom versus patient from the adapted features. That probe measures how much "phantomness" survives.
Analysis. The analytical core is two measurements. The first is an exchange rate: how many annotated real CT scans a fine-tuned or few-shot model needs before it matches the phantom-paired model. The second is a contrast-fidelity analysis. For each organ we will measure how closely phantom CT/MRI contrast ratios match patient contrast ratios. We predict that the per-organ Dice gain tracks this fidelity.
Baselines
The naive reference point is source-only nnU-Net trained on labeled MRI and applied directly to CT. M3DA reports Dice 0.032 for this MRI-to-CT shift (M3DA). We will reproduce it on the same splits to confirm our pipeline matches.
The strongest existing label-free method is entropy minimization with nnU-Net's heavy augmentation. M3DA reports Dice 0.388 on MRI-to-CT, the best of more than ten adaptation methods it benchmarked (M3DA). It is the right main competitor because it already uses both patient streams we use and differs only in lacking phantom pairs. We will run it with the same backbone, the same patient data, the same number of training iterations, and the same hyperparameter search budget. Beating it means a Dice improvement over 0.388 that is statistically significant across at least three seeds and patient-level bootstrap resampling. We also require average surface distance to improve and not merely Dice.
The strongest intermediate-domain competitor is MiDSS-style Fourier amplitude mixing and copy-paste between source MRI and target CT (MiDSS). It gained 13.57% Dice on prostate data, but only for shifts within one modality. It tests whether synthetic bridges built from unpaired patients can substitute for physically paired bridges.
For label cost, the comparator is Meta-hallucinator at 1, 4, and more annotated CT scans. It improved on the MT-UDA baseline by 6.7% Dice and 0.9 mm average surface distance in the 4-shot cardiac setting. This comparison yields the exchange rate. We will run it on MM-WHS as reported, and on the abdominal task with the same patient data.
Two ablations isolate the key ingredient. The first uses phantom scans as extra labeled data in both modalities but deletes the correspondence term. If this matches the full method, the value lies in cheap labeled target-modality images and not in pairing, which is a weaker but still useful finding. The second shuffles pair assignments across phantoms, so modality correspondence is broken while every other property of the data is kept.
Why not just…?
Why not just push augmentation and entropy minimization harder? M3DA already shows this recipe is the strongest in its benchmark and still stops at 0.388 on MRI-to-CT (M3DA). Intensity augmentation perturbs appearance, but it cannot tell the network which MRI intensity corresponds to which CT intensity when contrast orderings invert. Our main comparison runs exactly this recipe with and without the phantom stream, so any gain is attributable.
Why not just translate MRI to CT with CycleGAN or a diffusion model? M3DA found that image-translation methods including CycleGAN often did no better than simple self-training. Meta-hallucinator attributes this to translators lacking segmentation feedback (Meta-hallucinator). Paired phantoms would help a translator too, however. If a phantom-supervised translator matched our method, pairing would still be validated as the key resource. We will therefore include a translation baseline that also receives phantom pairs.
Why not just synthesize images from label maps with random intensities, with no phantom at all? This is the most serious threat. It gives free labels and forces invariance to contrast without building anything. The file contains no reported result for this on these benchmarks, so it cannot be dismissed on evidence. Our computational-phantom arm, with randomized-contrast synthesis as a further variant, tests it directly. If randomization matches physical phantoms, the conclusion is that correspondence to realistic contrast is unnecessary, and physical phantom construction is not worth its cost. We would report that outcome as the finding.
Why not just annotate four real CT scans? Four annotations may be cheap in some settings, and Meta-hallucinator shows they help substantially. The exchange-rate experiment answers this question directly. If the phantom stream is worth fewer than one annotated scan, it is not worth building. If it is worth several and also stacks with real shots, it earns its place.
Proposed Experiments
The primary experiment is the M3DA abdominal MRI-to-CT shift built on AMOS. We will report per-organ Dice and average surface distance for the source-only model, entropy minimization with augmentation, MiDSS-style mixing, the phantom translation baseline, the full method, and the two ablations. We will also run the reverse direction and the MM-WHS cardiac task against Meta-hallucinator, using computational cardiac phantoms because a physical cardiac phantom is out of scope.
Three checks are decisive. The first is whether the full method beats 0.388 while the shuffled-pair ablation does not. If both succeed or both fail, pairing is not the mechanism. The second is whether per-organ gain correlates with measured phantom contrast fidelity. A null correlation would suggest the network gains from something other than physics correspondence. The third is whether errors on patient CT concentrate in regions phantoms cannot represent, such as bowel gas, motion-blurred liver dome, and vessels, and whether the phantom-versus-patient probe accuracy stays near chance. Together these separate a method that learned modality physics from one that learned to see patients as phantoms.
References
- M3DA: Benchmark for Unsupervised Domain Adaptation in 3D Medical Image Segmentation
- Autonomous Robotic Tissue Palpation and Abnormalities Characterisation via Ergodic Exploration
- Rendering with a Gut Feeling: Depth-Guided Triangle Splatting for Physically Consistent Colonoscopic Reconstruction
- Fusing Prediction and Perception: Adaptive Kalman Filter-Driven Respiratory Gating for MR Surgical Navigation
- Meta-hallucinator: Towards Few-Shot Cross-Modality Cardiac Image Segmentation
- Constructing and Exploring Intermediate Domains in Mixed Domain Semi-supervised Medical Image Segmentation
- Impact of CT dose on AI performance: A comparison of radiomics, deep, and foundation models in a multicentric anthropomorphic phantom study
- A Novel Polymer-Encapsulated Multi-Imaging Modality Fiducial Marker with Positive Signal Contrast for Image-Guided Radiation Therapy