Paired Image Generation with Diffusion-Guided Diffusion Models

Haoxuan Zhang, Wenju Cui, Yuzhu Cao, Tao Tan, Jie Liu, Yunsong Peng, Jian ZhengarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces a novel "Paired Image Generation" (PIG) method using diffusion models to create both realistic medical images (specifically, digital breast tomosynthesis slices) and their corresponding segmentation masks. This approach addresses the critical shortage of annotated medical data by generating high-quality, paired synthetic data, which significantly improves the performance of downstream mass lesion segmentation tasks.

What Problem Does This Paper Address?

Early detection of breast cancer relies heavily on analyzing medical images like Digital Breast Tomosynthesis (DBT). However, dense breast tissue can obscure mass lesions, making them difficult for radiologists to annotate manually. This difficulty leads to a scarcity of high-quality annotated datasets, which are crucial for training robust deep learning models for mass lesion segmentation. Existing diffusion models, while good at generating images, struggle with generating accurate lesion details due to this concealment problem. Furthermore, they typically generate only images, not the corresponding annotations, limiting their direct utility for supervised training.

Methods

The core of this work is the proposed Paired Image Generation (PIG) method, built upon diffusion models. PIG tackles the limitations of existing methods by:

  • Unconditional Paired Generation: Unlike conditional diffusion models that require external inputs (like masks) to guide generation, PIG leverages the inherent paired relationship between an image and its mask as an internal guiding signal.
  • Diffusion Guider: The method mathematically proves that the generation of paired images can be achieved through two mutually guided diffusion processes. To implement this, they train an "additional diffusion guider" for a conditional diffusion model. This guider essentially learns to generate one part of the pair (e.g., the mask) to guide the generation of the other (e.g., the DBT slice), and vice-versa, without needing external conditions.
  • Dual Diffusion Processes: The PIG method models the generation of paired images (image $x$ and mask $y$) as two coupled Markovian diffusion processes. The forward process gradually adds noise to both $x$ and $y$, while the reverse process (the denoising part) is implemented using two diffusion models that guide each other. This mutual guidance is key to improving the quality of both the generated images and their corresponding masks.

Datasets

  • DBTMassSeg: A private dataset sourced from Suzhou Municipal Hospital and Guizhou Provincial People's Hospital. It contains 8,723 DBT slices with corresponding mass lesion masks, manually annotated by radiologists. The dataset includes images from CC and MLO perspectives. The authors preprocessed this data by cropping blank areas and resizing images to 512x512.

Key Insights and Ideas

  • Self-Guiding Generation: The most significant insight is the ability to generate high-quality paired images and masks without relying on external conditional inputs. The paired nature of the data itself becomes the guiding signal.
  • Improved Lesion Focus: The diffusion guider mechanism allows the model to pay more attention to the lesion areas, leading to more accurate and detailed generation of these critical regions, even when they are initially concealed in the original data.
  • Bridging the Annotation Gap: By generating paired images and masks, PIG directly addresses the scarcity of annotated data, a major bottleneck in medical AI development. This synthetic data can be seamlessly integrated into existing supervised training pipelines.

Key Results

  • Enhanced Generation Quality: PIG achieved a significantly lower Frechet Inception Distance (FID) score of 15.88 compared to unconditional diffusion models like DDPM (31.84) and DDIM (31.54), indicating superior image generation quality.
  • Improved Segmentation Performance: When the generated paired images were used to augment the training data for a mass lesion segmentation task, the PIG method led to substantial improvements. For instance, adding 2048 PIG-generated images to the dataset resulted in a Dice score of 60.54%, a significant jump from the baseline 52.36% and outperforming other conditional generation methods. The performance peaked around 2048 generated images, demonstrating the effectiveness of PIG's generated data.
  • Stability: The paper notes that PIG's performance dips to its lowest point after generating around 3072 images, suggesting a degree of stability and robustness in its generation process compared to other methods.

Limitations and Future Work

  • Annotation Dependency for Training: While PIG generates paired data without external conditions during generation, the training of the diffusion guider still relies on existing paired data (the DBTMassSeg dataset).
  • Optimal Data Augmentation Quantity: The study found that performance peaks around 2048 generated images, and increasing this number further can lead to fluctuations or dips. Determining the optimal quantity of synthetic data for different downstream tasks could be an area for future exploration.
  • Generalizability: While demonstrated for DBT mass segmentation, future work could explore the applicability of PIG to other medical imaging modalities and segmentation tasks where annotated data is scarce.

Summary updated

Abstract

The segmentation of mass lesions in digital breast tomosynthesis (DBT) images is very significant for the early screening of breast cancer. However, the high-density breast tissue often leads to high concealment of the mass lesions, which makes manual annotation difficult and time-consuming. As a result, there is a lack of annotated data for model training. Diffusion models are commonly used for data augmentation, but the existing methods face two challenges. First, due to the high concealment of lesions, it is difficult for the model to learn the features of the lesion area. This leads to the low generation quality of the lesion areas, thus limiting the quality of the generated images. Second, existing methods can only generate images and cannot generate corresponding annotations, which restricts the usability of the generated images in supervised training. In this work, we propose a paired image generation method. The method does not require external conditions and can achieve the generation of paired images by training an extra diffusion guider for the conditional diffusion model. During the experimental phase, we generated paired DBT slices and mass lesion masks. Then, we incorporated them into the supervised training process of the mass lesion segmentation task. The experimental results show that our method can improve the generation quality without external conditions. Moreover, it contributes to alleviating the shortage of annotated data, thus enhancing the performance of downstream tasks. The source code is available at https://github.com/zhanghx1320/PIG.
BibTeX
@misc{zhang2025paired,
  author = {Haoxuan Zhang and Wenju Cui and Yuzhu Cao and Tao Tan and Jie Liu and Yunsong Peng and Jian Zheng},
  title = {{Paired Image Generation with Diffusion-Guided Diffusion Models}},
  year = {2025},
  eprint = {2507.14833},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.14833},
}
Figure