Abstract
Deep learning models in medical imaging are often bottlenecked by the scarcity of high-quality, annotated multi-modal data. This paper introduces Paired Diffusion, a generative framework that simultaneously creates synthetic CT and PET scans alongside their corresponding ground-truth segmentation masks. By utilizing linked Denoising Diffusion Probabilistic Models (DDPMs) and a novel "noise-seeding" strategy, the framework generates massive, fully-annotated datasets, effectively bypassing the traditional costs and privacy constraints associated with manual medical image labeling.
Introduction
Radiotherapy planning and other complex medical tasks require vast amounts of annotated data. However, acquiring such data is difficult due to HIPAA/GDPR privacy concerns and the extreme cost of expert manual annotation. While standard generative models like GANs can create realistic images, they often fail to generate the associated labels (segmentation masks) or maintain spatial consistency across different modalities (e.g., ensuring a tumor appears in the same location on both a synthetic PET and CT scan). Paired Diffusion addresses these challenges by ensuring structural alignment across multiple generated channels.
Methodology
The authors propose a multi-network architecture where each modality (CT, PET, and Segmentation Mask) is handled by a dedicated DDPM. These networks are synchronized through several key mechanisms:
- Conditional Encoders & CBAM: Sub-networks use conditional encoders to share features. A Convolutional Block Attention Module (CBAM) dynamically adjusts the influence of these features based on the diffusion timestep.
- Timestep-Controlled Low-Pass (TLP) Filter: To clean conditional inputs, the authors developed a learnable frequency-space filter. Unlike previous complex filters, the TLP filter uses a simpler two-variable approach to remove noise while preserving semantic structures like organ shapes.
- Noise Seeding: Rather than starting from pure noise, the process begins by adding 80-90% noise to a real image. This "seeds" the generation with a realistic global structure while allowing the model to produce unique, synthetic details.
- Modified Perceptual Loss: A VGG-16 based perceptual loss forces the networks to align high-level features, preventing artifacts like "bright" bone in CT from being misinterpreted as tumor signals in the PET channel.
Key Results
The study utilized the HaN-Seg and HECKTOR 2022 datasets for evaluation. Key findings include:
- Improved Convergence: Downstream segmentation models (e.g., TransUNet) trained on synthetic data converged significantly faster.
- Accuracy Gains: Pre-training on synthetic paired data led to a statistically significant improvement in segmentation accuracy ($p = 0.00046$) in data-constrained scenarios.
- Qualitative Success: The "Seeded" approach produced synthetic images that maintained perfect spatial alignment between CT, PET, and Mask channels, a major hurdle for previous DDPM implementations.
Discussion and Future Work
While Paired Diffusion shows great promise, the authors noted that perceptual loss must be carefully tuned to avoid misaligning unrelated bright features. Additionally, training multiple DDPMs simultaneously is computationally expensive. Future research will focus on scaling the model to full 3D consistency for clinical radiotherapy applications and exploring faster sampling methods like GANs or VAEs to provide the initial structural "seed."
Summary updated