MedicalDPO: Multi-Objective Direct Preference Optimization for CT to PET Translation

Huiting Zhang, Yingxin Cai, Siyuan Wang, Xi Yang, Yuyao Yan, Rui Zhang, Kaizhu HuangarXiv 2026 · arXiv

TL;DR

MedicalDPO is a new framework that significantly improves CT-to-PET image synthesis by teaching diffusion models to "prefer" higher-quality outputs. By ranking multiple generated candidates across spatial, structural, and intensity dimensions and using a regularized fine-tuning approach, it achieves state-of-the-art results in recovering functional metabolic details from anatomical scans.

What Problem Does This Paper Address?

Synthesizing PET (Positron Emission Tomography) images from CT (Computed Tomography) scans is a "holy grail" in medical imaging. CT shows anatomy (bones and organs), while PET shows function (metabolic activity, like tumors "lighting up"). Because PET reflects information not fully visible in CT, the translation is incredibly difficult.

Current state-of-the-art methods use Brownian Bridge Diffusion Models (BBDMs). While powerful, these models are stochastic—meaning if you give them one CT scan, they can generate multiple different PET versions. Some of these versions are excellent, while others might miss a lesion or get the intensity wrong. Standard training ignores this "quality hierarchy," treating every generated sample the same way. Furthermore, applying standard preference optimization (like DPO used in LLMs) to medical images often causes the model to "drift," destroying the absolute anatomical fidelity required for clinical use.

Methods

The authors introduce MedicalDPO, which treats stochastic generation as a feature rather than a bug. Their approach consists of three main pillars:

  1. Multi-Objective Preference Construction: Instead of relying on a single score, they evaluate generated PET candidates across three dimensions:
  2. Spatial Fidelity: Using Dice coefficients to check if high-uptake regions (like tumors) are in the right place.
  3. Structural Fidelity: Using SSIM to ensure the overall "look" matches the ground truth.
  4. Intensity Fidelity: Using Mean Absolute Error (MAE) and a novel Attention-Weighted MAE (WMAE) to ensure the metabolic values are accurate.

    A "preferred" image must outperform a "rejected" image across all these metrics to be used for training.

  5. Regularized Preference Optimization: To prevent the model from degrading during fine-tuning, they jointly train the DPO objective with the original Brownian Bridge reconstruction loss. This "anchors" the model to the ground-truth mapping while it learns to prefer better-quality variations.

  6. Condition-Balanced Multi-Pair Rotation: Since some CT scans might produce many valid "preferred-rejected" pairs while others produce few, the model rotates through different pairs across training epochs to ensure no single patient condition overpowers the learning process.
Figure 2. Overview of the proposed MedicalDPO framework for CT-to-PET synthesis, built on Brownian Bridge diffusion models [12, 16]. The figure visualizes the corresponding CT and PET images, while c denotes the latent CT conditioning representation used by the Brownian Bridge model. Multiple generated PET candidates are evaluated against the ground-truth PET using complementary spatial, structural, and intensity criteria to construct reliable preference pairs. The selected preferred and rejected samples are encoded as yw and yl, forming the latent preference triplet (c, yw, yl) for optimizing the trainable policy πθ against the frozen reference model πref with L = LDPO + λLreg. Open/closed locks denote trainable/frozen components.

Datasets

The study utilized two large-scale, publicly available whole-body PET/CT datasets:

  • FDG-PET-CT-Lesions: Focused on tumor lesions.
  • ViMed-PET: A diverse clinical dataset. Combined, these provided 111,630 paired CT and PET slices for training and evaluation.

Key Insights and Ideas

  • Medical Quality is a Vector, Not a Scalar: Figure 1 demonstrates that an image might look structurally perfect but have the wrong intensity, or vice versa. By requiring a "clean sweep" across multiple metrics, MedicalDPO ensures the preference signal is robust and clinically relevant.
  • The "Intra-Condition Hierarchy" is Free Supervision: Every time a diffusion model generates a "bad" sample alongside a "good" one for the same input, it creates a learning opportunity. MedicalDPO mines this existing data without needing extra human labeling.
  • DPO Needs a "Guardrail" in Medicine: The authors found that "DPO-only" fine-tuning (the standard in NLP) actually makes medical models worse because it loses track of absolute pixel values. The regularization term ($\lambda\mathcal{L}_{reg}$) is essential for stability.
Figure 1. Different fidelity criteria select preferred and rejected PET candidates from 20 CPDM-generated candidates for the same CT condition. The full strategy combines spatial, structural, and intensity fidelity for more reliable preference construction.

Key Results

MedicalDPO was tested on two different diffusion backbones (BBDM and CPDM) and consistently outperformed them:

  • BBDM Improvements: Reduced perceptual error (LPIPS) by 11.67% and intensity error (MAE) by 13.22%.
  • CPDM Improvements: Achieved a 5.23% reduction in MAE, showing that even "stronger" base models benefit from preference alignment.
  • Qualitative Gains: As shown in Figure 3, the model is significantly better at recovering coherent structures and local uptake patterns (the "hot spots" in PET) compared to the base models or GAN-based baselines like Pix2Pix.
Figure 3. Qualitative comparison of CT-to-PET synthesis methods. The green box highlights the overall improvement of BBDM [12] after applying MedicalDPO, while the red boxes emphasize local improvements over CPDM [16], better matching the ground-truth PET.

Limitations and Future Work

  • 2D vs. 3D: The current work operates on 2D slices. The authors acknowledge that moving to native 3D translation would better capture volumetric anatomical context.
  • Clinical Endpoints: While the model improves image metrics (SSIM, PSNR), future work needs to evaluate "downstream" clinical tasks, such as whether this improves a radiologist's ability to detect lesions.
  • Weighting Preferences: Currently, all metrics are weighted equally. Future iterations could explore "medically informed" weighting, perhaps prioritizing intensity accuracy in known tumor regions.

Summary updated

Abstract

CT-to-PET synthesis estimates functional PET images from anatomical CT scans for auxiliary analysis, representation pretraining, and missing-modality completion. Brownian Bridge diffusion models can produce multiple PET candidates of different quality under the same CT condition, exposing an intra-condition quality hierarchy. Candidate quality spans complementary spatial, structural, and intensity dimensions, while DPO-only fine-tuning may compromise absolute synthesis fidelity. We propose MedicalDPO which converts this hierarchy into preference supervision automatically constructed from paired CT-PET training data. Multiple candidates are ranked across the three fidelity dimensions, and strict cross-metric verification retains preferred-rejected pairs with consistent quality separation. Relative denoising-error preference optimization is jointly trained with the original Brownian Bridge objective to anchor the pretrained CT-to-PET mapping, while condition-balanced multi-pair rotation distributes preference supervision across CT conditions. Experiments on BBDM and CPDM show improvements across LPIPS, MAE, SSIM, and PSNR. Further analyses demonstrate metric complementarity and clear pair separation, while ablations show that the original training objective prevents the substantial degradation caused by DPO-only fine-tuning.
BibTeX
@misc{zhang2026medicaldpo,
  author = {Huiting Zhang and Yingxin Cai and Siyuan Wang and Xi Yang and Yuyao Yan and Rui Zhang and Kaizhu Huang},
  title = {{MedicalDPO: Multi-Objective Direct Preference Optimization for CT to PET Translation}},
  year = {2026},
  eprint = {2611.21098},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.21098},
}
Figure