Auto-GAN: Self-Supervised Collaborative Learning for Medical Image Synthesis

Bing Cao, Han Zhang, Nannan Wang, Xinbo Gao, Dinggang Shen

Introduction

In clinical diagnostics, multi-modal imaging—such as various MRI sequences (T1, T2, FLAIR) or CT scans—is essential for comprehensive tissue characterization. However, clinical datasets are frequently incomplete due to time constraints, patient discomfort, or cost. While Generative Adversarial Networks (GANs) have shown promise in cross-modality synthesis, they often struggle with structural distortion and fail to leverage the complementary information present across multiple available modalities.

Methodology: The Auto-GAN Framework

Auto-GAN addresses these limitations through a collaborative learning paradigm. The architecture comprises three primary components: a Translation Network, a Self-Representation Network, and a Discriminator.

The Auto-GAN architecture utilizes a teacher-student framework where a self-representation network provides feature-level guidance to the translation network.

Translation and Self-Representation

The Translation Network ($T$) acts as the "student," utilizing a multi-branch encoder to process any combination of available modalities. These features are fused in a latent space before being decoded into the target modality. To ensure structural integrity, the Self-Representation Network ($S$) acts as a "teacher." Trained as an auto-encoder to reconstruct the target modality from itself, it learns a robust, "clean" feature representation of the target domain.

Collaborative Learning

During training, the decoder features of the teacher network supervise those of the student network. This feature-level constraint forces the student to learn the underlying anatomical distribution of the target modality, preventing the blurring and deformation common in models relying solely on pixel-level loss functions ($L_1$ or $L_2$).

Experimental Results

The framework was evaluated on the BraTS (Brain Tumor Segmentation) and ADNI (Alzheimer’s Disease Neuroimaging Initiative) databases, as well as natural image datasets for generalizability.

Quantitative Performance

Auto-GAN significantly outperformed baseline models like Pix2Pix and CycleGAN. On the BraTS dataset, the model achieved a Structural Similarity Index (SSIM) of 0.9157 and a Feature-Similarity Index (FSIM) of 0.9592, compared to baseline SSIM scores that typically ranged between 0.77 and 0.88.

Comparison of T2 synthesis results using different input combinations. The collaborative approach (f) utilizing T1, T1-C, and T2-FLAIR achieves superior structural accuracy and higher metrics.

Qualitative Analysis

Qualitative assessments revealed that Auto-GAN preserves critical anatomical details, such as skull contours in MRI-to-CT tasks and the specific textures of brain lesions. Unlike standard GANs that may "hallucinate" artifacts or blur tumor boundaries, Auto-GAN maintains high fidelity to the ground truth by effectively distilling information from all available input branches.

Conclusion

Auto-GAN introduces a robust method for medical image synthesis by combining multi-modal inputs with self-supervised feature guidance. By moving beyond simple pixel-to-pixel translation, the framework ensures that synthesized images are not only visually realistic but also anatomically accurate, making it a valuable tool for clinical decision support where data may be missing.

Summary updated

Abstract

In various clinical scenarios, medical image is crucial in disease diagnosis and treatment. Different modalities of medical images provide complementary information and jointly helps doctors to make accurate clinical decision. However, due to clinical and practical restrictions, certain imaging modalities may be unavailable nor complete. To impute missing data with adequate clinical accuracy, here we propose a framework called self-supervised collaborative learning to synthesize missing modality for medical images. The proposed method comprehensively utilize all available information correlated to the target modality from multi-source-modality images to generate any missing modality in a single model. Different from the existing methods, we introduce an auto-encoder network as a novel, self-supervised constraint, which provides target-modality-specific information to guide generator training. In addition, we design a modality mask vector as the target modality label. With experiments on multiple medical image databases, we demonstrate a great generalization ability as well as specialty of our method compared with other state-of-the-arts.
BibTeX
@inproceedings{cao2020auto,
  author = {Bing Cao and Han Zhang and Nannan Wang and Xinbo Gao and Dinggang Shen},
  title = {{Auto-GAN: Self-Supervised Collaborative Learning for Medical Image Synthesis}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i07.6619},
}
Figure