Unsupervised Tumor-Aware Distillation for Multi-Modal Brain Image Translation

Chuan Huang, Jia Wei, Rui LiIJCNN 2024

Introduction

In clinical neurology, multi-modal MRI sequences such as T1, T2, and Flair are essential for comprehensive brain tumor assessment. However, clinical realities often result in "missing modalities" due to time constraints, costs, or image artifacts. While unsupervised image-to-image translation offers a solution to synthesize missing scans, standard methods frequently suffer from tumor deformation, blurring or distorting critical pathological details.

UTAD-Net (Unsupervised Tumor-Aware Distillation Network) addresses this by introducing a framework that preserves tumor fidelity without requiring manual masks during the inference (prediction) phase, making it a viable tool for automated clinical workflows.

Methods

The core of UTAD-Net is a Teacher-Student distillation strategy designed to transfer tumor-aware knowledge from a mask-informed model to a mask-free one.

Overview of the UTAD-Net architecture including the Teacher-Student distillation and cycle training process.
  1. The Teacher Network: This network utilizes a dual-branch architecture: a Global Branch for the whole image and a Local Branch that focuses specifically on the tumor area (extracted using a mask). During training, the teacher learns an end-to-end mapping using unpaired images and their corresponding tumor masks.
  2. The Student Network: The student shares the same architecture but is intentionally "handicapped"—it is never shown the tumor mask.
  3. Knowledge Distillation: The student is trained to mimic the intermediate features and the final output of the teacher. Through this distillation, the student learns to "perceive" tumor areas internally.
  4. Local Consistency Loss: A specific loss function ensures that the tumor area in the synthesized "whole image" matches the tumor area in the "local branch," preventing the deformation common in other unsupervised models.

Datasets

The study utilizes the BraTS 2020 (Brain Tumor Segmentation) dataset, which provides multi-modal scans and expert-segmented tumor masks.

Example of the four MRI modalities (Flair, T1, T1ce, and T2) from the BraTS dataset.
  • Modality Mix: Includes Flair, T1, T1ce, and T2 scans.
  • Annotations: Features expert-segmented masks for three regions: Whole Tumor (WT), Tumor Core (TC), and Enhancing Tumor (ET).
  • Setup: 180 samples for training, 30 for validation, and 159 for testing. All images were resized to 128 × 128 pixels.

Key Results and Insights

UTAD-Net was compared against established baselines like StarGAN, DRIT++, and TarGAN, demonstrating superior performance in both image quality and pathological accuracy.

  • Image Quality: UTAD-Net achieved a PSNR of 31.48 for the student network, significantly higher than StarGAN’s 28.02. It also showed superior SSIM (Structural Similarity) scores.
  • Tumor Fidelity: In "Local SSIM" (measuring only the tumor area), UTAD-Net outperformed all baselines, proving it preserves the lesion's structure better than standard translation models.
  • Clinical Utility: When synthesized images were fed into a downstream segmentation tool (nnU-Net), the Dice Similarity Coefficient (DSC) for the Tumor Core (TC) was 0.7184, compared to StarGAN’s 0.6357. This indicates the generated tumors are realistic enough for other AI tools to recognize them correctly.

Conclusion and Future Work

UTAD-Net successfully moves the requirement for tumor masks from the inference stage to the training stage via distillation. By forcing a student to mimic a mask-aware teacher, the model develops an internal "attention" for tumors.

While the current implementation is limited to 2D slices and requires masks for training the teacher, it represents a significant step toward robust, automated clinical tools for multi-modal MRI synthesis. Future work may involve extending the architecture to 3D convolutional approaches to maintain volumetric spatial consistency.

Summary updated

BibTeX
@inproceedings{huang2024unsupervised,
  author = {Chuan Huang and Jia Wei and Rui Li},
  title = {{Unsupervised Tumor-Aware Distillation for Multi-Modal Brain Image Translation}},
  booktitle = {IJCNN},
  year = {2024},
}
Figure