TL;DR
MedicalDPO is a new framework that significantly improves CT-to-PET image synthesis by teaching diffusion models to "prefer" higher-quality outputs. By ranking multiple generated candidates across spatial, structural, and intensity dimensions and using a regularized fine-tuning approach, it achieves state-of-the-art results in recovering functional metabolic details from anatomical scans.
What Problem Does This Paper Address?
Synthesizing PET (Positron Emission Tomography) images from CT (Computed Tomography) scans is a "holy grail" in medical imaging. CT shows anatomy (bones and organs), while PET shows function (metabolic activity, like tumors "lighting up"). Because PET reflects information not fully visible in CT, the translation is incredibly difficult.
Current state-of-the-art methods use Brownian Bridge Diffusion Models (BBDMs). While powerful, these models are stochastic—meaning if you give them one CT scan, they can generate multiple different PET versions. Some of these versions are excellent, while others might miss a lesion or get the intensity wrong. Standard training ignores this "quality hierarchy," treating every generated sample the same way. Furthermore, applying standard preference optimization (like DPO used in LLMs) to medical images often causes the model to "drift," destroying the absolute anatomical fidelity required for clinical use.
Methods
The authors introduce MedicalDPO, which treats stochastic generation as a feature rather than a bug. Their approach consists of three main pillars:
- Multi-Objective Preference Construction: Instead of relying on a single score, they evaluate generated PET candidates across three dimensions:
- Spatial Fidelity: Using Dice coefficients to check if high-uptake regions (like tumors) are in the right place.
- Structural Fidelity: Using SSIM to ensure the overall "look" matches the ground truth.
-
Intensity Fidelity: Using Mean Absolute Error (MAE) and a novel Attention-Weighted MAE (WMAE) to ensure the metabolic values are accurate.
A "preferred" image must outperform a "rejected" image across all these metrics to be used for training.
-
Regularized Preference Optimization: To prevent the model from degrading during fine-tuning, they jointly train the DPO objective with the original Brownian Bridge reconstruction loss. This "anchors" the model to the ground-truth mapping while it learns to prefer better-quality variations.
- Condition-Balanced Multi-Pair Rotation: Since some CT scans might produce many valid "preferred-rejected" pairs while others produce few, the model rotates through different pairs across training epochs to ensure no single patient condition overpowers the learning process.
Datasets
The study utilized two large-scale, publicly available whole-body PET/CT datasets:
- FDG-PET-CT-Lesions: Focused on tumor lesions.
- ViMed-PET: A diverse clinical dataset. Combined, these provided 111,630 paired CT and PET slices for training and evaluation.
Key Insights and Ideas
- Medical Quality is a Vector, Not a Scalar: Figure 1 demonstrates that an image might look structurally perfect but have the wrong intensity, or vice versa. By requiring a "clean sweep" across multiple metrics, MedicalDPO ensures the preference signal is robust and clinically relevant.
- The "Intra-Condition Hierarchy" is Free Supervision: Every time a diffusion model generates a "bad" sample alongside a "good" one for the same input, it creates a learning opportunity. MedicalDPO mines this existing data without needing extra human labeling.
- DPO Needs a "Guardrail" in Medicine: The authors found that "DPO-only" fine-tuning (the standard in NLP) actually makes medical models worse because it loses track of absolute pixel values. The regularization term ($\lambda\mathcal{L}_{reg}$) is essential for stability.
Key Results
MedicalDPO was tested on two different diffusion backbones (BBDM and CPDM) and consistently outperformed them:
- BBDM Improvements: Reduced perceptual error (LPIPS) by 11.67% and intensity error (MAE) by 13.22%.
- CPDM Improvements: Achieved a 5.23% reduction in MAE, showing that even "stronger" base models benefit from preference alignment.
- Qualitative Gains: As shown in Figure 3, the model is significantly better at recovering coherent structures and local uptake patterns (the "hot spots" in PET) compared to the base models or GAN-based baselines like Pix2Pix.
Limitations and Future Work
- 2D vs. 3D: The current work operates on 2D slices. The authors acknowledge that moving to native 3D translation would better capture volumetric anatomical context.
- Clinical Endpoints: While the model improves image metrics (SSIM, PSNR), future work needs to evaluate "downstream" clinical tasks, such as whether this improves a radiologist's ability to detect lesions.
- Weighting Preferences: Currently, all metrics are weighted equally. Future iterations could explore "medically informed" weighting, perhaps prioritizing intensity accuracy in known tumor regions.
Summary updated