Improving the generalizability of convolutional neural network-based segmentation on CMR images

Chen Chen, Wenjia Bai, Rhodri H. Davies, Anish N. Bhuva, Charlotte Manisty, James C. Moon, Nay Aung, Aaron M. Lee, Mihir M. Sanghvi, Kenneth Fung, José Miguel Paiva, Steffen E. Petersen, Elena Lukaschuk, Stefan K. Piechnik, Stefan Neubauer, Daniel RueckertarXiv 2019 · DOI · OpenReview

Abstract

Convolutional neural network (CNN) based segmentation methods provide an efficient and automated way for clinicians to assess the structure and function of the heart in cardiac MR images. While CNNs can generally perform the segmentation tasks with high accuracy when training and test images come from the same domain (e.g. same scanner or site), their performance often degrades dramatically on images from different scanners or clinical sites. We propose a simple yet effective way for improving the network generalization ability by carefully designing data normalization and augmentation strategies to accommodate common scenarios in multi-site, multi-scanner clinical imaging data sets. We demonstrate that a neural network trained on a single-site single-scanner dataset from the UK Biobank can be successfully applied to segmenting cardiac MR images across different sites and different scanners without substantial loss of accuracy. Specifically, the method was trained on a large set of 3,975 subjects from the UK Biobank. It was then directly tested on 600 different subjects from the UK Biobank for intra-domain testing and two other sets for cross-domain testing: the ACDC dataset (100 subjects, 1 site, 2 scanners) and the BSCMR-AS dataset (599 subjects, 6 sites, 9 scanners). The proposed method produces promising segmentation results on the UK Biobank test set which are comparable to previously reported values in the literature, while also performing well on cross-domain test sets, achieving a mean Dice metric of 0.90 for the left ventricle, 0.81 for the myocardium and 0.82 for the right ventricle on the ACDC dataset; and 0.89 for the left ventricle, 0.83 for the myocardium on the BSCMR-AS dataset. The proposed method offers a potential solution to improve CNN-based model generalizability for the cross-scanner and cross-site cardiac MR image segmentation task.

The Challenge of Domain Shift in Cardiac Imaging

Convolutional Neural Networks (CNNs) have revolutionized cardiac structure segmentation, provided the test data matches the training distribution. However, real-world clinical environments present a significant challenge known as "domain shift." Images from different hospitals or scanner manufacturers (e.g., Siemens vs. Philips) vary significantly in noise levels, contrast, and spatial resolution.

Overview of the domain shift problem: models trained on a single source domain (A) often fail to generalize when deployed to different clinical domains (B, C, N).

Traditionally, this is addressed via fine-tuning on new labeled data or complex Unsupervised Deep Domain Adaptation (UDDA), both of which are difficult to scale. This research explores whether a "universal" cardiac segmentation model can be built using data from just one scanner that generalizes across diverse clinical sites without further tuning.

Methods: Prioritizing the Pipeline over Complexity

The study utilizes a standard 2D U-Net architecture, chosen for its memory efficiency and robustness against varying slice thickness and respiratory motion artifacts. Rather than introducing complex new loss functions, the authors focus on a disciplined pre-processing and augmentation pipeline:

  1. Image Resampling: Standardizing all images to a median resolution of 1.25 × 1.25 mm to ensure consistent heart scale.
  2. Intensity Normalization: Normalizing images to a mean of 0 and standard deviation of 1.
  3. Heavy Data Augmentation: Employing aggressive random rotations (±30°), scaling (0.7x to 1.4x), and flips during training.
  4. Network Capacity: Utilizing wider networks (e.g., UNet-64) which were found to generalize significantly better than lightweight versions by providing more "room" to learn variations.

Datasets for Generalization Testing

To rigorously evaluate generalization, the model was trained on the UK Biobank (UKBB) source domain (3,975 subjects, single site, 1.5T Siemens scanner) and tested on:

  • UKBB Test Set: 600 subjects for intra-domain evaluation.
  • ACDC Dataset: 100 subjects from France using 1.5T and 3T Siemens scanners, including various pathologies.
  • BSCMR-AS Dataset: 599 subjects from 6 UK hospitals using 9 different scanner types (Philips and Siemens), representing a highly heterogeneous real-world test.

Key Results and Performance

The proposed model demonstrated remarkable stability across unseen datasets without any retraining. It significantly outperformed previous benchmarks, particularly in segmenting the myocardium and right ventricle, which are traditionally more difficult.

Comparison of segmentation performance (Dice metric) between the proposed method and a previous benchmark (Bai et al., 2018) across UKBB, ACDC, and BSCMR-AS datasets.

On the cross-domain ACDC dataset, the model achieved a mean Dice metric of 0.90 for the left ventricle (LV) and 0.81 for the myocardium. On the multi-site BSCMR-AS dataset, it achieved a Dice of 0.89 for the LV and 0.83 for the myocardium. Bland-Altman analysis further confirmed high agreement between automated measurements and manual expert annotations across these diverse sites.

Insights and Future Directions

The findings suggest that image resampling and high network capacity are critical for robustness. While the model handles scanner variations well, it still faces challenges with extreme morphological changes (e.g., severe Hypertrophic Cardiomyopathy) not well-represented in the training set. Future work may involve deformation-based data augmentation to synthesize pathological heart shapes, further reducing the need for massive rare-disease data collection.

Summary updated

Abstract

Convolutional neural network (CNN) based segmentation methods provide an efficient and automated way for clinicians to assess the structure and function of the heart in cardiac MR images. While CNNs can generally perform the segmentation tasks with high accuracy when training and test images come from the same domain (e.g. same scanner or site), their performance often degrades dramatically on images from different scanners or clinical sites. We propose a simple yet effective way for improving the network generalization ability by carefully designing data normalization and augmentation strategies to accommodate common scenarios in multi-site, multi-scanner clinical imaging data sets. We demonstrate that a neural network trained on a single-site single-scanner dataset from the UK Biobank can be successfully applied to segmenting cardiac MR images across different sites and different scanners without substantial loss of accuracy. Specifically, the method was trained on a large set of 3,975 subjects from the UK Biobank. It was then directly tested on 600 different subjects from the UK Biobank for intra-domain testing and two other sets for cross-domain testing: the ACDC dataset (100 subjects, 1 site, 2 scanners) and the BSCMR-AS dataset (599 subjects, 6 sites, 9 scanners). The proposed method produces promising segmentation results on the UK Biobank test set which are comparable to previously reported values in the literature, while also performing well on cross-domain test sets, achieving a mean Dice metric of 0.90 for the left ventricle, 0.81 for the myocardium and 0.82 for the right ventricle on the ACDC dataset; and 0.89 for the left ventricle, 0.83 for the myocardium on the BSCMR-AS dataset. The proposed method offers a potential solution to improve CNN-based model generalizability for the cross-scanner and cross-site cardiac MR image segmentation task.
BibTeX
@misc{chen2019improving,
  author = {Chen Chen and Wenjia Bai and Rhodri H. Davies and Anish N. Bhuva and Charlotte Manisty and James C. Moon and Nay Aung and Aaron M. Lee and Mihir M. Sanghvi and Kenneth Fung and José Miguel Paiva and Steffen E. Petersen and Elena Lukaschuk and Stefan K. Piechnik and Stefan Neubauer and Daniel Rueckert},
  title = {{Improving the generalizability of convolutional neural network-based segmentation on CMR images}},
  year = {2019},
  eprint = {1907.01268},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1907.01268},
}
Figure