Automating Carotid Intima-Media Thickness Video Interpretation with Convolutional Neural Networks

Jae Y. Shin, Nima Tajbakhsh, R. Todd Hurst, Christopher B. Kendall, Jianming Liang

Abstract

This paper introduces the first unified framework to fully automate the interpretation of Carotid Intima-Media Thickness (CIMT) ultrasound videos. By automating frame selection, region-of-interest (ROI) localization, and boundary tracing using Convolutional Neural Networks (CNNs), the system achieves human-level accuracy. This automation addresses the labor-intensive nature of CIMT assessment, potentially moving a powerful cardiovascular risk tool into routine clinical practice.

Introduction

Cardiovascular disease (CVD) remains the leading cause of mortality globally. While preventable, the challenge lies in identifying at-risk individuals early. Measuring the thickness of the inner layers of the carotid artery (CIMT) via ultrasound is a proven, non-invasive way to predict CVD risk.

Anatomy of a carotid ultrasound image showing the Intima-Media Thickness (IMT) and the Carotid Bulb landmark.

However, manual CIMT measurement is a significant burden for clinicians. A sonographer must sift through video to find specific end-diastolic frames, locate a precise 1cm section of the artery distal to the "carotid bulb," and meticulously trace layer interfaces to measure thickness in micrometers. This labor-intensive process is prone to human error, limiting the widespread clinical use of CIMT. The authors propose a robust, automated pipeline to replace this manual workflow.

Methodology

The authors propose a three-stage pipeline powered by a unified CNN framework (specifically, a LeNet-like architecture):

  1. Automated Frame Selection: Instead of analyzing the artery directly for timing, the system processes the ECG signal embedded in the ultrasound video. Using "accumulated difference images" to filter noise, a 2-way CNN identifies the R-peaks of the heart cycle, corresponding to the required end-diastolic frames.
  2. Constrained ROI Localization: Locating the correct 1cm patch of the artery is difficult because the vessel looks similar along its length. The authors utilize a 3-way CNN to identify both the ROI and the "carotid bulb"—a distinct anatomical landmark. They then apply a "contextual constraint": since clinical guidelines dictate the ROI should be approximately 1cm from the bulb, the bulb's location is used to refine and anchor the ROI's position.
  3. CIMT Measurement: A 3-way CNN classifies pixels as the lumen-intima interface, the media-adventitia interface, or background. To ensure smooth and continuous boundaries, the CNN output initializes "open snakes" (active contour models) that snap to the true anatomical boundaries at a sub-pixel level.
Comparison of unconstrained vs. constrained ROI localization using the carotid bulb as a spatial anchor.

Key Insights and Results

The system's success relies on two primary innovations: leveraging clinical domain knowledge (using the bulb as an anchor) and a hybrid approach that combines the discriminative power of deep learning with the geometric constraints of active contours.

Validation on a dataset of 92 CIMT videos showed that the system significantly outperformed previous state-of-the-art hand-crafted algorithms:

  • Thickness Accuracy: The system achieved a mean absolute error of 2.8 ± 2.1 μm, a massive improvement over the 13.8 ± 31.9 μm error seen in previous automated methods.
  • Localization Precision: The constrained method reduced ROI localization error to 0.19 mm, compared to 0.35 mm for an unconstrained CNN.
  • Clinical Reliability: The error margin is well below the 400 μm difference typically used to distinguish between healthy and high-risk populations, making the tool clinically viable.

Conclusion and Future Work

By automating the most tedious aspects of CIMT interpretation, this framework removes a major barrier to cardiovascular screening. While the current model relies on an embedded ECG signal for frame selection, future iterations could explore ECG-free timing. Additionally, while the simple LeNet-like architecture proved effective, future work may explore more modern architectures like U-Nets for further segmentation refinement.

Summary updated

Abstract

Cardiovascular disease (CVD) is the leading cause of mortality yet largely preventable, but the key to prevention is to identify at-risk individuals before adverse events. For predicting individual CVD risk, carotid intima-media thickness (CIMT), a noninvasive ultrasound method, has proven to be valuable, offering several advantages over CT coronary artery calcium score. However, each CIMT examination includes several ultrasound videos, and interpreting each of these CIMT videos involves three operations: (1) select three end-diastolic ultrasound frames (EUF) in the video, (2) localize a region of interest (ROI) in each selected frame, and (3) trace the lumen-intima interface and the media-adventitia interface in each ROI to measure CIMT. These operations are tedious, laborious, and time consuming, a serious limitation that hinders the widespread utilization of CIMT in clinical practice. To overcome this limitation, this paper presents a new system to automate CIMT video interpretation. Our extensive experiments demonstrate that the suggested system significantly outperforms the state-of-the-art methods. The superior performance is attributable to our unified framework based on convolutional neural networks (CNNs) coupled with our informative image representation and effective post-processing of the CNN outputs, which are uniquely designed for each of the above three operations.
BibTeX
@inproceedings{shin2016automating,
  author = {Jae Y. Shin and Nima Tajbakhsh and R. Todd Hurst and Christopher B. Kendall and Jianming Liang},
  title = {{Automating Carotid Intima-Media Thickness Video Interpretation with Convolutional Neural Networks}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2016},
  url = {https://openreview.net/forum?id=H1NFSlMuWH},
}
Figure