3D Consistent Biventricular Myocardial Segmentation Using Deep Learning for Mesh Generation

Qiao Zheng, Hervé Delingette, Nicolas Duchateau, Nicholas AyachearXiv 2018 · arXiv

TL;DR

This paper introduces a deep learning framework for 3D-consistent segmentation of the biventricular myocardium from MRI volumes. By using a two-stage approach—initial slice segmentation followed by spatial propagation—the method ensures the geometric consistency required for direct 3D mesh generation, even when trained on very small datasets.

What Problem Does This Paper Address?

Accurate 3D modeling of the heart is essential for biophysical simulations and clinical diagnosis. However, most existing automated methods focus on 2D slice-by-slice segmentation of either the left or right ventricle in isolation.

This approach has two major drawbacks:

  1. Lack of 3D Consistency: Independent 2D segmentations often don't "line up" perfectly across slices, making it difficult to generate the smooth, watertight meshes needed for simulations.
  2. Apex Failure: Slices near the bottom of the heart (the apex) often have poor image quality, causing standard 2D models to fail completely in these regions.

The authors aim to bridge this gap by creating a model-free, fast, and data-efficient pipeline that produces spatially consistent biventricular (BV) segmentations.

Methods

The authors propose a pipeline centered around two distinct convolutional neural networks (CNNs), both utilizing a multi-scale, coarse-to-fine architecture.

Fig. 1: Overview of the proposed method. (1) Segmentation of a single slice at the middle of the volume by the initialization network. (2) Propagation of predicted mask towards base and apex by the spatial propagation network.

1. The Two-Network Architecture

  • Initialization Net: This network segments a single, high-quality slice from the middle of the heart volume.
  • Spatial Propagation Net: This is the "secret sauce" for consistency. It takes a segmented slice, its predicted mask, and the next four raw image slices as input to predict the masks for those next four slices. This process iterates upward toward the base and downward toward the apex.

2. Multi-Scale Sub-Networks

Both networks are composed of sub-networks (SubNet32, SubNet64, and SubNet128) that process data at increasing resolutions. This allows the model to capture broad anatomical context at low resolutions and refine edges at higher resolutions.

The detailed architecture of the Initialization and Spatial Propagation networks, showing the multi-scale coarse-to-fine structure.

3. Stabilized and Class-Balanced Loss

To handle the fact that the myocardium occupies only about 20% of the image (class imbalance) and to prevent numerical instability (like log(0) errors), the authors designed a custom loss function. It essentially stops "pushing" the network to improve pixels that are already correctly classified with high confidence, allowing the optimizer to focus on the difficult boundary pixels.

Datasets

Training deep learning models usually requires thousands of images, but medical data is scarce.

  • Primary Dataset: The authors used only 15 annotated subjects from the STACOM database.
  • Data Augmentation: To compensate for the small sample size, they used a motion simulation and image synthesis algorithm to generate 11,160 synthetic volumes. These variants included simulations of both healthy and infarcted (heart attack) tissue to increase the model's robustness.

Key Insights and Ideas

  • Propagation > Independent Segmentation: By treating segmentation as a propagation task rather than a series of independent 2D tasks, the model "remembers" the shape of the heart from previous slices. This makes it incredibly robust in the apex, where image contrast is often too low for standard models to work.
  • Data Efficiency through Synthesis: The paper demonstrates that a small clinical dataset can be successfully leveraged by using biophysical models to generate realistic synthetic training data.
  • Model-Free Simplicity: Unlike "shape-constrained" models that require complex mathematical priors about heart geometry, this method is purely data-driven, making it easier for other researchers to implement.

Key Results

  • Speed: The entire volume segmentation takes approximately 3 seconds on a GPU, matching the fastest state-of-the-art methods.
  • Accuracy: The method achieved a 3D Dice index (a measure of overlap between predicted and ground truth) of approximately 0.785.
  • Consistency: Qualitative results showed that the propagation method maintained smooth transitions across slices, whereas segmenting slices independently led to total failure near the apex. The resulting masks were clean enough to be used directly for mesh generation via standard tools like CGAL.

Limitations and Future Work

  • Manual Initialization: While the segmentation is automated, a human still needs to manually identify the "base" slice (the top of the ventricles) to tell the propagation where to stop.
  • Temporal Consistency: This paper focuses on spatial consistency (3D). The authors note that the next step is ensuring temporal consistency—making sure the heart's shape moves smoothly across the frames of a video (4D).
  • Dataset Size: While the synthetic data helped, the authors expect significant performance gains if the model is trained on a larger variety of real-world clinical cases.

Summary updated

Abstract

We present a novel automated method to segment the myocardium of both left and right ventricles in MRI volumes. The segmentation is consistent in 3D across the slices such that it can be directly used for mesh generation. Two specific neural networks with multi-scale coarse-to-fine prediction structure are proposed to cope with the small training dataset and trained using an original loss function. The former segments a slice in the middle of the volume. Then the latter iteratively propagates the slice segmentations towards the base and the apex, in a spatially consistent way. We perform 5-fold cross-validation on the 15 cases from STACOM to validate the method. For training, we use real cases and their synthetic variants generated by combining motion simulation and image synthesis. Accurate and consistent testing results are obtained.
BibTeX
@misc{zheng2018d,
  author = {Qiao Zheng and Hervé Delingette and Nicolas Duchateau and Nicholas Ayache},
  title = {{3D Consistent Biventricular Myocardial Segmentation Using Deep Learning for Mesh Generation}},
  year = {2018},
  eprint = {1803.11080},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1803.11080},
}

Loading related content...

Figure