OF-MSRN: Optical Flow-Auxiliary Multi-Task Regression Network for Direct Quantitative Measurement, Segmentation and Motion Estimation

Chengqian Zhao, Cheng Feng, Dengwang Li, Shuo Li

Introduction

Monitoring the carotid artery is a cornerstone of cardiovascular disease diagnosis. Two primary clinical metrics are used: Carotid Artery Lumen Diameter (CALD) and Carotid Intima-Media Thickness (CIMT). Automating these measurements via ultrasound imaging is challenging due to the thinness of the intima-media layer, motion artifacts from arterial pulsing, and pathological complexities like plaques that obscure vessel walls.

Clinical metrics for carotid artery ultrasound: CALD (Carotid Artery Lumen Diameter) and CIMT (Carotid Intima-Media Thickness).

Traditional automated methods often treat measurement as a post-processing step following segmentation, which can lead to error propagation. The OF-MSRN (Optical Flow-Auxiliary Multi-Task Regression Network) addresses these issues by integrating segmentation, quantitative measurement, and motion estimation into a single, unified framework.

Methodology

The OF-MSRN architecture is designed to leverage temporal consistency across ultrasound sequences to enhance measurement accuracy. It consists of three primary components:

The OF-MSRN architecture, featuring the Direct Quantitative Measurement and Segmentation (DQMS) branch and the Motion Estimation (ME) branch linked by the Optical Flow-Auxiliary (OFA) module.

1. Multi-scale Feature Encoder (MFE)

The backbone of the network is a Siamese-style encoder utilizing stacked ResNext blocks. This structure extracts both low-level and high-level features, ensuring that the minute structural details of the CIMT are captured effectively.

2. Direct Quantitative Measurement and Segmentation (DQMS)

Unlike standard segmentation networks that perform pixel-wise classification, the DQMS branch treats vessel boundaries as a series of discrete coordinates. This regression-based approach is more robust to the shape variations and thin structures typical of the carotid artery.

3. Unsupervised Motion Estimation (ME) and the OFA Module

The ME branch employs a Bidirectional Optical Flow (BOF) module to predict pixel movement between frames without requiring ground truth motion labels. The Optical Flow-Auxiliary (OFA) module then links these tasks. Using a ConvLSTM and differentiable inverse warping, it uses predicted motion to "warp" segmentations between frames. Discrepancies between warped and actual frames serve as a penalty, forcing the network to refine both its motion estimation and its structural segmentation simultaneously.

Experimental Results

The model was validated on a clinical dataset of 101 patients, using 202 synthetic ultrasound sequences (45 frames each) covering at least one cardiac cycle. Expert physician annotations served as the ground truth.

Quantitative Performance

OF-MSRN demonstrated superior performance compared to baseline models:

  • Measurement Accuracy: It achieved a Mean Absolute Error (MAE) of 1.23 pixels for CALD and 0.93 pixels for CIMT, outperforming the next best method (MRNns).
  • Segmentation Quality: The model reached an Intersection over Union (IoU) of 0.908 for CALD and 0.893 for CIMT, a significant improvement over standard FCN-based approaches.

Key Insights

  • Co-Promotion: The integration of motion estimation provides a "temporal anchor," helping the network ignore transient artifacts and motion blur caused by the arterial pulse.
  • Regression Robustness: Treating segmentation as a coordinate regression problem allows for better handling of the extremely thin CIMT layer compared to traditional classification.
  • Bidirectional Consistency: Calculating optical flow in both directions helps the model identify and ignore outliers, such as occlusions caused by diseased tissue.

Conclusion

OF-MSRN represents a significant step forward in automated carotid artery analysis. By treating segmentation and motion estimation as mutually beneficial tasks, the framework overcomes the inherent limitations of ultrasound imaging. Future work will focus on validating the model across other imaging modalities (MRI/CT) and optimizing the computational efficiency of the multi-task architecture for real-time clinical applications.

Summary updated

BibTeX
@inproceedings{zhao2020msrn,
  author = {Chengqian Zhao and Cheng Feng and Dengwang Li and Shuo Li},
  title = {{OF-MSRN: Optical Flow-Auxiliary Multi-Task Regression Network for Direct Quantitative Measurement, Segmentation and Motion Estimation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i01.5475},
}
Figure