Fully Automatic Myocardial Segmentation of Contrast Echocardiography Sequence Using Random Forests Guided by Shape Model

Yuanwei Li, Chin Pang Ho, Matthieu Toulemonde, Navtej Chahal, Roxy Senior, Meng-Xing Tang

TL;DR

This paper introduces a fully automatic pipeline for segmenting the myocardium in noisy Myocardial Contrast Echocardiography (MCE) sequences. By integrating a statistical shape model directly into a Random Forest classifier and adding a temporal consistency constraint, the authors achieve state-of-the-art accuracy that overcomes the heavy noise and misleading artifacts typical of contrast ultrasound.

What Problem Does This Paper Address?

Myocardial Contrast Echocardiography (MCE) is a powerful tool for detecting coronary artery disease because it uses microbubbles to visualize blood flow (perfusion) within the heart muscle. However, analyzing these images is a nightmare for traditional computer vision.

The images suffer from high speckle noise, signal attenuation, and "shadowing." Furthermore, structures like papillary muscles often have the same intensity as the myocardium, leading standard classifiers to produce "leaky" or physically impossible segmentations. While Random Forests (RF) are great at local pixel classification, they are "shape-blind"—they don't understand that the myocardium must form a specific, coherent structure. Manual segmentation by clinicians is the current gold standard, but it is slow, subjective, and prone to observer fatigue.

Methods

The authors propose a multi-stage pipeline that moves from coarse localization to fine, shape-constrained segmentation:

  1. CNN-based Bounding Box Detection: First, a Convolutional Neural Network (based on CaffeNet) automatically locates the heart within the larger ultrasound frame. This "rigid alignment" step removes variations in pose, scale, and rotation, simplifying the task for subsequent steps.
  2. Random Forest with Shape Model (SM) Features: This is the core innovation. Standard RFs look at local pixel intensities. The authors introduce a novel SM feature: for any given pixel, the feature calculates the signed distance to a "plausible" myocardial shape sampled from a statistical model. This forces the RF to "learn" what a heart looks like globally while it classifies pixels locally.
  3. Statistical Shape Model (SSM) Fitting: The output of the RF is a probability map. To ensure the final result is anatomically valid, the authors fit a PCA-based shape model to this map. This acts as a sophisticated "filter" that ignores outliers and fills in gaps where the ultrasound signal might be weak.
  4. Temporal Consistency (2D+t): For video sequences, they add a temporal constraint to the optimization. This ensures the heart shape transitions smoothly from one frame to the next, preventing the "jitter" often seen in frame-by-frame automated methods.

Datasets

The study utilized clinical MCE data from 21 subjects (10 healthy, 11 with coronary artery disease).

  • Dataset1: 242 static 2D images across different apical views and cardiac phases (end-systole and end-diastole).
  • Dataset2: 12 full-cycle 2D+t sequences (averaging 22 frames each). All data was manually annotated by an expert to provide a ground truth for training and evaluation.

Key Insights and Ideas

  • Feature-Level Integration: Most researchers use shape models only as a post-processing step. By including shape information as a feature within the Random Forest, the classifier becomes aware of global geometry during the initial decision-making process.
  • The "Papillary Muscle" Solution: Because the shape model knows the myocardium is a continuous wall, the system successfully ignores internal structures (like papillary muscles) that look like heart tissue but are in the "wrong" place.
  • Pose Normalization Matters: Using a CNN to crop the image significantly boosted performance. It allows the subsequent RF and Shape Model to focus on a standardized coordinate space rather than hunting for the heart across the entire noisy frame.

Key Results

  • Accuracy: The proposed method (M2/N3 in the paper) significantly outperformed classic RF, Active Shape Models (ASM), and image registration techniques. It achieved a higher Jaccard Index (overlap) and lower Mean Absolute Distance (MAD).
  • Clinical Correlation: The automated measurements of endocardial and myocardial areas showed a high correlation ($r=0.93$ and $r=0.69$ respectively) with manual expert measurements.
  • Speed: The entire pipeline takes approximately 9.2 seconds per image on standard hardware. While not yet "real-time" (30fps), it is fast enough for near-instantaneous clinical review after a scan.

Limitations and Future Work

  • Dataset Size: While 21 subjects provided enough data for a proof-of-concept, larger, more diverse datasets are needed to ensure the model generalizes across different ultrasound machines and patient populations.
  • 3D Extension: The current work is 2D+t. Extending this to 3D (volumetric) ultrasound would be the logical next step, though it would significantly increase the computational cost of the shape model fitting.
  • Optimization Bottleneck: The shape model fitting is the slowest part of the pipeline. The authors suggest that future work could involve predicting shape parameters directly from the image to bypass the iterative optimization step.

Summary updated

Abstract

Myocardial contrast echocardiography (MCE) is an imaging technique that assesses left ventricle function and myocardial perfusion for the detection of coronary artery diseases. Automatic MCE perfusion quantification is challenging and requires accurate segmentation of the myocardium from noisy and time-varying images. Random forests (RF) have been successfully applied to many medical image segmentation tasks. However, the pixel-wise RF classifier ignores contextual relationships between label outputs of individual pixels. RF which only utilizes local appearance features is also susceptible to data suffering from large intensity variations. In this paper, we demonstrate how to overcome the above limitations of classic RF by presenting a fully automatic segmentation pipeline for myocardial segmentation in full-cycle 2-D MCE data. Specifically, a statistical shape model is used to provide shape prior information that guide the RF segmentation in two ways. First, a novel shape model (SM) feature is incorporated into the RF framework to generate a more accurate RF probability map. Second, the shape model is fitted to the RF probability map to refine and constrain the final segmentation to plausible myocardial shapes. We further improve the performance by introducing a bounding box detection algorithm as a preprocessing step in the segmentation pipeline. Our approach on 2-D image is further extended to 2-D+t sequences which ensures temporal consistency in the final sequence segmentations. When evaluated on clinical MCE data sets, our proposed method achieves notable improvement in segmentation accuracy and outperforms other state-of-the-art methods, including the classic RF and its variants, active shape model and image registration.
BibTeX
@article{li2018fully,
  author = {Yuanwei Li and Chin Pang Ho and Matthieu Toulemonde and Navtej Chahal and Roxy Senior and Meng-Xing Tang},
  title = {{Fully Automatic Myocardial Segmentation of Contrast Echocardiography Sequence Using Random Forests Guided by Shape Model}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2018},
  url = {https://openreview.net/forum?id=P50wM-RJGH},
}

Loading related content...

Figure