Cooperative Training and Latent Space Data Augmentation for Robust Medical Image Segmentation

Chen Chen, Kerstin Hammernik, Cheng Ouyang, Chen Qin, Wenjia Bai, Daniel Rueckert

Abstract

Deep learning models for medical image segmentation often struggle with domain shifts caused by variations in scanners and imaging protocols. This paper proposes a "Thinking, Fast and Slow" framework that decouples anatomical shape from image appearance. By employing a cooperative training strategy between a Fast-thinking Network (FTN) and a Slow-thinking Network (STN), along with a novel latent space data augmentation technique, the model achieves significant robustness improvements. Experimental results on cardiac MRI datasets demonstrate a 15% increase in segmentation accuracy under low-data regimes and superior generalization to unseen clinical environments and artifacts.

Introduction: The Challenge of Domain Shift

Medical image segmentation models frequently exhibit "brittleness" when deployed in real-world clinical settings. A model trained on high-quality MRI scans from one institution may fail when encountering data from another hospital due to differences in scanner hardware, contrast settings, or common artifacts like motion blur. This phenomenon, known as domain shift, typically requires large, diverse datasets to overcome—data that is often difficult to acquire due to privacy and cost constraints. This research addresses the challenge of training a robust model using limited data from a single site that can generalize to diverse, unseen environments.

Methodology: A "Thinking, Fast and Slow" Framework

The proposed architecture is inspired by dual-process psychological theories, specifically the "Thinking, Fast and Slow" model.

Fast and Slow Thinking Networks

The framework consists of two primary components:

  1. Fast-Thinking Network (FTN): Acting as the "intuitive" system, the FTN performs simultaneous image reconstruction and initial segmentation. It incorporates a feature decoupler to isolate anatomical shape features from stylistic image features (e.g., texture and brightness).
  2. Slow-Thinking Network (STN): Serving as the "logical" system, the STN is a denoising autoencoder that refines the FTN's initial segmentation by applying learned anatomical shape priors.
Overview of the proposed framework consisting of the Fast-thinking Network (FTN), Slow-thinking Network (STN), and the hard example generation process via latent code masking.

Latent Space Data Augmentation

To enhance robustness, the authors introduce a novel data augmentation strategy that operates in the latent space rather than the image space. By manipulating the compressed mathematical representations of images, the system generates "hard examples" through three masking schemes:

  • Random Dropout: Stochastic removal of latent code elements.
  • Targeted Channel-wise Masking: Identifying and masking the most predictive features to force the network to utilize secondary cues.
  • Targeted Spatial-wise Masking: Simulating occlusions by masking specific anatomical regions.

This approach allows the model to learn from a wide variety of synthetic corruptions, preparing it for real-world artifacts.

Experimental Evaluation

The framework was evaluated on cardiac MRI segmentation tasks focusing on the ventricles and myocardium. Training was conducted on a minimal subset of the ACDC dataset (only 10 subjects). Testing involved:

  • Intra-domain: Standard ACDC test set.
  • Cross-domain: The M&Ms dataset, featuring data from five different clinical sites with varying vendors and diseases.
  • Robustness: ACDC-C, a synthetic benchmark simulating MR artifacts such as bias fields, ghosting, motion, and spikes.

Key Results and Discussion

The cooperative framework demonstrated substantial performance gains over standard training methods.

Performance comparison (AVG Dice) across intra-domain, cross-domain, and unseen corruption datasets, highlighting the robustness of the cooperative FTN+STN approach.

Key findings include:

  • Significant Accuracy Gains: The proposed method achieved a 15% improvement in average Dice scores compared to standard baselines in low-data scenarios.
  • Enhanced Generalization: The model significantly outperformed existing methods on the multi-site M&Ms dataset, proving its ability to handle cross-vendor domain shifts despite being trained on data from only one site.
  • Artifact Resilience: The system showed high stability against simulated artifacts, particularly "spikes" and "ghosting," which typically degrade standard segmentation performance.

Conclusion and Future Directions

By explicitly decoupling style from structure and utilizing latent space augmentation, this framework provides a powerful solution for robust medical image segmentation with limited training data. While highly effective for cardiac structures with consistent shapes, future work will explore the application of this "shape prior" approach to more complex and variable structures, such as tumors. Additionally, extending the framework to other modalities like CT and Ultrasound remains a promising avenue for research.

Summary updated

Abstract

Deep learning-based segmentation methods are vulnerable to unforeseen data distribution shifts during deployment, e.g. change of image appearances or contrasts caused by different scanners, unexpected imaging artifacts etc. In this paper, we present a cooperative framework for training image segmentation models and a latent space augmentation method for generating hard examples. Both contributions improve model generalization and robustness with limited data. The cooperative training framework consists of a fast-thinking network (FTN) and a slow-thinking network (STN). The FTN learns decoupled image features and shape features for image reconstruction and segmentation tasks. The STN learns shape priors for segmentation correction and refinement. The two networks are trained in a cooperative manner. The latent space augmentation generates challenging examples for training by masking the decoupled latent space in both channel-wise and spatial-wise manners. We performed extensive experiments on public cardiac imaging datasets. Using only 10 subjects from a single site for training, we demonstrated improved cross-site segmentation performance, and increased robustness against various unforeseen imaging artifacts compared to strong baseline methods. Particularly, cooperative training with latent space data augmentation yields 15% improvement in terms of average Dice score when compared to a standard training method.
BibTeX
@inproceedings{chen2021cooperative,
  author = {Chen Chen and Kerstin Hammernik and Cheng Ouyang and Chen Qin and Wenjia Bai and Daniel Rueckert},
  title = {{Cooperative Training and Latent Space Data Augmentation for Robust Medical Image Segmentation}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2021},
  url = {https://openreview.net/forum?id=q__AWYTcCF},
}
Figure