Automatic 3D bi-ventricular segmentation of cardiac images by a shape-constrained multi-task deep learning approach

Jinming Duan, Ghalib Bello, Jo Schlemper, Wenjia Bai, Timothy J. W. Dawes, Carlo Biffi, Antonio de Marvao, Georgia Doumou, Declan P. O'Regan, Daniel RueckertarXiv 2018 · arXiv

Introduction

Cardiac Magnetic Resonance (CMR) imaging is the gold standard for assessing cardiac function. However, clinical CMR data often suffers from low resolution, thick slices (e.g., 10mm), and inter-slice shift artifacts caused by respiratory motion. Standard automated segmentation methods frequently produce "staircase" effects—jagged, anatomically implausible 3D reconstructions. This research presents a novel pipeline that integrates multi-task deep learning with anatomical shape priors to produce high-fidelity, smooth 3D models of the heart's ventricles.

Methodology

The proposed architecture, the Simultaneous Segmentation and Landmark Localisation Network (SSLLN), adopts a 2.5D approach. By treating 3D volumes as multi-channel images, the network leverages inter-slice spatial context without the memory constraints of full 3D convolutions. The SSLLN performs two tasks in parallel: pixel-wise segmentation and the localization of six key anatomical landmarks (such as the apex and mitral valve center).

The proposed automated pipeline, showing the integration of the SSLLN and the shape refinement process.

Following initial segmentation, a Shape Refinement stage uses the detected landmarks as robust anchors to align high-resolution anatomical "atlases" to the patient's specific scan. Through non-rigid registration and label fusion, the system smooths artifacts and "inpaints" missing data, ensuring the final model adheres to known anatomical constraints.

Experimental Setup and Results

The model was validated on two extensive datasets: the UK Digital Heart Project (1,831 healthy subjects, high-resolution) and a Pulmonary Hypertension (PH) dataset (649 subjects, low-resolution). The inclusion of PH patients, whose hearts are significantly deformed, tested the model's robustness to pathological variations.

Key findings include:

  • Accuracy: The SSLLN achieved a Dice score of 0.96 for the left ventricular cavity on high-resolution data, surpassing standard 2D and 3D FCN baselines.
  • Robustness: The system successfully handled the severe anatomical deformations in PH patients, proving it can adapt beyond healthy heart shapes.
  • Landmark Precision: AI landmark detection error was found to be lower than the inter-observer variability between human experts.
Quantitative performance comparison (Dice Index and Hausdorff Distance) of the proposed SSLLN-HR against 2D and 3D FCN baselines.

Discussion and Future Work

While the current two-stage process is highly accurate and removes artifacts that plague standard AI methods, the shape refinement step introduces a computational overhead of approximately 15-20 minutes. Future iterations could focus on end-to-end training—combining segmentation and refinement into a single network—and utilizing GPU-accelerated registration to further streamline the workflow for real-time clinical use.

Summary updated

Abstract

Deep learning approaches have achieved state-of-the-art performance in cardiac magnetic resonance (CMR) image segmentation. However, most approaches have focused on learning image intensity features for segmentation, whereas the incorporation of anatomical shape priors has received less attention. In this paper, we combine a multi-task deep learning approach with atlas propagation to develop a shape-constrained bi-ventricular segmentation pipeline for short-axis CMR volumetric images. The pipeline first employs a fully convolutional network (FCN) that learns segmentation and landmark localisation tasks simultaneously. The architecture of the proposed FCN uses a 2.5D representation, thus combining the computational advantage of 2D FCNs networks and the capability of addressing 3D spatial consistency without compromising segmentation accuracy. Moreover, the refinement step is designed to explicitly enforce a shape constraint and improve segmentation quality. This step is effective for overcoming image artefacts (e.g. due to different breath-hold positions and large slice thickness), which preclude the creation of anatomically meaningful 3D cardiac shapes. The proposed pipeline is fully automated, due to network's ability to infer landmarks, which are then used downstream in the pipeline to initialise atlas propagation. We validate the pipeline on 1831 healthy subjects and 649 subjects with pulmonary hypertension. Extensive numerical experiments on the two datasets demonstrate that our proposed method is robust and capable of producing accurate, high-resolution and anatomically smooth bi-ventricular 3D models, despite the artefacts in input CMR volumes.
BibTeX
@misc{duan2018automatic,
  author = {Jinming Duan and Ghalib Bello and Jo Schlemper and Wenjia Bai and Timothy J. W. Dawes and Carlo Biffi and Antonio de Marvao and Georgia Doumou and Declan P. O'Regan and Daniel Rueckert},
  title = {{Automatic 3D bi-ventricular segmentation of cardiac images by a shape-constrained multi-task deep learning approach}},
  year = {2018},
  eprint = {1808.08578},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1808.08578},
}
Figure