3D Consistent & Robust Segmentation of Cardiac Images by Deep Learning with Spatial Propagation

Qiao Zheng, Hervé Delingette, Nicolas Duchateau, Nicholas AyachearXiv 2018 · arXiv

TL;DR

This paper introduces a deep learning framework for cardiac MRI segmentation that enforces 3D spatial consistency by propagating information slice-by-slice. By using the segmentation of an adjacent slice as contextual input for the next, the model mimics human expert behavior, achieving state-of-the-art robustness and impressive generalization across diverse clinical datasets without the heavy computational burden of full 3D convolutional networks.

What Problem Does This Paper Address?

Automated cardiac segmentation in MRI is a cornerstone of modern cardiology, yet it faces a persistent architectural dilemma.

Standard 2D methods are computationally efficient and benefit from large amounts of training data (treating each slice as a sample), but they are "blind" to the 3D context. This often leads to spatial inconsistencies, such as the model failing to detect the heart on difficult apical slices or producing jagged, unrealistic volumes. Conversely, 3D methods theoretically capture this context but are plagued by high memory requirements, the need for data downsampling, and a relative scarcity of full 3D labeled volumes for training.

Furthermore, cardiac MRI often has anisotropic resolution—slices are thick (5-10mm) compared to their in-plane resolution (1-2mm). This makes standard 3D convolutions less effective. The authors aim to bridge this gap with a method that retains 2D efficiency while explicitly enforcing 3D consistency.

Methods

The authors propose a two-stage pipeline:

  1. ROI-net: A variant of U-Net designed to perform coarse heart/background segmentation to define a Region of Interest (ROI). This saves memory and focuses the subsequent segmentation step.
  2. Segmentation with Propagation (LVRV-net and LV-net): This is the core innovation. Instead of segmenting a slice $S[i]$ in isolation, the network takes a 5-channel input: the current slice $S[i]$, the adjacent slice above it $S[i-1]$, and the predicted segmentation mask of that adjacent slice $M[i-1]$.
The architecture of ROI-net and the propagation-based LVRV-net.

By processing the stack iteratively from the base (top) to the apex (bottom), the model "propagates" its findings downward. This ensures that the prediction for the current slice is physically plausible given what was found just millimeters above it. The networks are trained using a multi-class Dice loss to handle class imbalance between the blood pools, myocardium, and background.

Datasets

The study stands out for its scale and breadth of evaluation:

  • UK Biobank: A massive dataset used for primary training (3,078 cases) and testing (756 cases).
  • ACDC (Automatic Cardiac Diagnosis Challenge): 100 cases used to test generalization to different pathologies (dilated cardiomyopathy, etc.).
  • Sunnybrook: 30 cases from the MICCAI 2009 challenge.
  • RVSC (Right Ventricle Segmentation Challenge): 16 cases from the MICCAI 2012 challenge.

Crucially, the model was tested on the latter three datasets without any fine-tuning, serving as a rigorous test of its "out-of-the-box" generalization ability.

Key Insights and Ideas

  • Mimicking Human Expertise: Human experts don't segment slices in a vacuum; they maintain consistency across the stack. The propagation mechanism explicitly builds this human-like bias into the architecture.
  • Generalization through Scale: By training on the vast UK Biobank, the model learns a robust "pattern" of cardiac anatomy that generalizes remarkably well to other cohorts, even when those cohorts have different labeling conventions (e.g., how the right ventricle is handled at the outflow tract).
  • The Hausdorff Advantage: While the model's Dice scores are comparable to other state-of-the-art (SOTA) methods, it significantly outperforms them in Hausdorff distance (a measure of the maximum error between contours). This indicates that the propagation method is much less likely to make "catastrophic" spatial errors.

Key Results

  • Robustness on "Difficult" Slices: The propagation method correctly identifies the heart on faint apical slices where standard 2D methods often fail completely.
  • 3D Consistency: Qualitative results show that the reconstructed 3D meshes from this method are smoother and more anatomically regular than those derived from human expert segmentations, which can sometimes be jittery between slices.
Comparison of LVRV-net vs. a non-propagating baseline on difficult apical slices. The propagation method (middle column) correctly tracks the apex where the baseline (right column) fails.
  • Cross-Dataset Performance: On the ACDC dataset, the model achieved a mean Hausdorff distance for the Left Ventricle (LV) comparable to SOTA methods specifically tuned for that challenge, despite being trained on a different population. On the RVSC dataset, it achieved a mean Hausdorff distance of 7.56mm, significantly better than the 8.86mm - 11.51mm range of other reported methods.

Limitations and Future Work

  • Ground-Truth Variability: The authors noted that the UK Biobank ground truth itself has high variability, particularly in the myocardium. This likely capped their Dice scores, as the model might be "correct" but penalized for not matching a biased manual label.
  • Base Slice Dependency: The method relies on identifying the "base" slice to start propagation. While they proposed an automated way to find this, errors at the very top of the stack could theoretically propagate downward.
  • Temporal Consistency: While this paper solves for 3D spatial consistency, the authors suggest that future work should extend this propagation logic to the temporal dimension (across the cardiac cycle) to ensure segmentations are consistent through time as well as space.

Summary updated

Abstract

We propose a method based on deep learning to perform cardiac segmentation on short axis MRI image stacks iteratively from the top slice (around the base) to the bottom slice (around the apex). At each iteration, a novel variant of U-net is applied to propagate the segmentation of a slice to the adjacent slice below it. In other words, the prediction of a segmentation of a slice is dependent upon the already existing segmentation of an adjacent slice. 3D-consistency is hence explicitly enforced. The method is trained on a large database of 3078 cases from UK Biobank. It is then tested on 756 different cases from UK Biobank and three other state-of-the-art cohorts (ACDC with 100 cases, Sunnybrook with 30 cases, RVSC with 16 cases). Results comparable or even better than the state-of-the-art in terms of distance measures are achieved. They also emphasize the assets of our method, namely enhanced spatial consistency (currently neither considered nor achieved by the state-of-the-art), and the generalization ability to unseen cases even from other databases.
BibTeX
@misc{zheng2018d,
  author = {Qiao Zheng and Hervé Delingette and Nicolas Duchateau and Nicholas Ayache},
  title = {{3D Consistent \& Robust Segmentation of Cardiac Images by Deep Learning with Spatial Propagation}},
  year = {2018},
  eprint = {1804.09400},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1804.09400},
}

Loading related content...

Figure