Introduction
In clinical diagnostics, multi-modal imaging—such as various MRI sequences (T1, T2, FLAIR) or CT scans—is essential for comprehensive tissue characterization. However, clinical datasets are frequently incomplete due to time constraints, patient discomfort, or cost. While Generative Adversarial Networks (GANs) have shown promise in cross-modality synthesis, they often struggle with structural distortion and fail to leverage the complementary information present across multiple available modalities.
Methodology: The Auto-GAN Framework
Auto-GAN addresses these limitations through a collaborative learning paradigm. The architecture comprises three primary components: a Translation Network, a Self-Representation Network, and a Discriminator.
Translation and Self-Representation
The Translation Network ($T$) acts as the "student," utilizing a multi-branch encoder to process any combination of available modalities. These features are fused in a latent space before being decoded into the target modality. To ensure structural integrity, the Self-Representation Network ($S$) acts as a "teacher." Trained as an auto-encoder to reconstruct the target modality from itself, it learns a robust, "clean" feature representation of the target domain.
Collaborative Learning
During training, the decoder features of the teacher network supervise those of the student network. This feature-level constraint forces the student to learn the underlying anatomical distribution of the target modality, preventing the blurring and deformation common in models relying solely on pixel-level loss functions ($L_1$ or $L_2$).
Experimental Results
The framework was evaluated on the BraTS (Brain Tumor Segmentation) and ADNI (Alzheimer’s Disease Neuroimaging Initiative) databases, as well as natural image datasets for generalizability.
Quantitative Performance
Auto-GAN significantly outperformed baseline models like Pix2Pix and CycleGAN. On the BraTS dataset, the model achieved a Structural Similarity Index (SSIM) of 0.9157 and a Feature-Similarity Index (FSIM) of 0.9592, compared to baseline SSIM scores that typically ranged between 0.77 and 0.88.
Qualitative Analysis
Qualitative assessments revealed that Auto-GAN preserves critical anatomical details, such as skull contours in MRI-to-CT tasks and the specific textures of brain lesions. Unlike standard GANs that may "hallucinate" artifacts or blur tumor boundaries, Auto-GAN maintains high fidelity to the ground truth by effectively distilling information from all available input branches.
Conclusion
Auto-GAN introduces a robust method for medical image synthesis by combining multi-modal inputs with self-supervised feature guidance. By moving beyond simple pixel-to-pixel translation, the framework ensures that synthesized images are not only visually realistic but also anatomically accurate, making it a valuable tool for clinical decision support where data may be missing.
Summary updated