Deep Learning Based Automatic Left Ventricle Segmentation from the Transgastric Short-Axis View on Transesophageal Echocardiography: A Feasibility Study

Yuan Tian, Wenting Qin, Zihang Zhao, Chunrong Wang, Yajie Tian, Yuelun Zhang, Kai He, Yuguan Zhang, Le Shen, Zhuhuang Zhou, Chunhua YuDiagnostics 2024 · DOI

TL;DR

This study explores the use of deep learning, specifically U-Net variants, for automatically segmenting the left ventricle from transgastric short-axis view (TSV) echocardiograms. The research demonstrates the feasibility of this approach, showing comparable accuracy to manual segmentation and offering a potential for faster, more objective cardiovascular assessments during surgery.

What Problem Does This Paper Address?

Accurate assessment of the left ventricle's structure and function is crucial for managing patients during surgery. Transesophageal echocardiography (TEE), particularly the transgastric short-axis view (TSV), provides excellent visualization for this purpose. However, manually segmenting the left ventricle from these images is a time-consuming and expertise-dependent task, which can be a bottleneck in critical perioperative settings. Existing deep learning approaches have primarily focused on transthoracic echocardiography (TTE) or different cardiac views, leaving a gap in understanding their efficacy for TSV TEE.

Methods

The researchers investigated four deep neural network architectures for left ventricle segmentation: U-Net, UNet++, Attention U-Net, and UNeXt. These models, all based on the U-Net architecture, were trained on a dataset of TSV TEE images. The process involved:

  • Image Preprocessing: Converting 3-channel grayscale images to 1-channel and resizing them to 256x256 pixels.
  • Data Augmentation: Applying random rotation, horizontal flipping, and vertical flipping to the training set to improve model robustness and prevent overfitting.
  • Model Training: Utilizing a combined loss function of binary cross-entropy (BCE) and Dice loss (LDSC) with a batch size of 4 and 100 training epochs.
  • Evaluation Metrics: Assessing segmentation performance using the Dice Similarity Coefficient (DSC) and Jaccard Similarity Coefficient (JSC), alongside metrics like parameter count, training time, and inference time.

Datasets

The study utilized a retrospective dataset of 1388 TSV TEE images collected from 451 patients who underwent perioperative TEE between July 2015 and October 2023. The dataset was split into:

  • Training Set: 1112 images (later augmented to 3336 images).
  • Validation Set: 138 images.
  • Test Set: 138 images.

Crucially, the dataset was curated to exclude images with known abnormalities, duplication, or significant noise, aiming for a representative sample of standard perioperative TEE scans. The manual segmentation performed by two anesthesiologists and confirmed by a senior anesthesiologist served as the ground truth.

Key Insights and Ideas

  • Feasibility Demonstrated: The study conclusively shows that deep learning, particularly U-Net variants, is a feasible and effective method for automatic left ventricle segmentation in TSV TEE images.
  • Comparable Performance: All tested U-Net models achieved high and comparable segmentation accuracy (DSC around 0.91-0.92), suggesting that even the basic U-Net can perform well on this task.
  • Attention Mechanism Advantage: The Attention U-Net model showed superior performance in challenging cases, such as those with impaired left ventricle boundaries or papillary muscle artifacts, highlighting the benefit of attention mechanisms in focusing on relevant image features.
  • Efficiency Trade-offs: Different models offer distinct advantages: UNeXt is the most parameter-efficient, while U-Net is the fastest in terms of training and inference time. This suggests a need to balance accuracy, speed, and computational resources based on specific application requirements.
  • Potential for Perioperative Workflow: The ability to automate segmentation could significantly speed up cardiovascular assessment during surgery, potentially leading to more timely clinical decisions.

Key Results

  • Accuracy: The Attention U-Net and U-Net++ models achieved the highest average JSC (86.02%) and DSC (92.00%) on the test set.
  • Efficiency:
    • UNeXt had the smallest number of model parameters (1.47 million).
  • U-Net had the shortest training time (6428.65 s) and the fastest inference time for a single image (101.75 ms).
  • Challenging Cases: Attention U-Net demonstrated better performance in segmenting left ventricles with boundary missing or papillary muscle interference compared to other models.
  • Statistical Significance: A Kruskal-Wallis test found no statistically significant differences in average JSC and DSC between the tested algorithms, reinforcing their comparable overall performance.

Limitations and Future Work

  • Challenging Cases: While Attention U-Net showed improvement, the study acknowledges that segmentation in very challenging cases (e.g., severe boundary missing, significant papillary muscle interference) remains an area for further research.
  • Dataset Size: Although described as a "large dataset" for TEE, it is smaller than some benchmark datasets for TTE (e.g., CAMUS). The authors suggest that further research is needed to explore the application of these models in more complex scenarios.
  • Real-time Assessment: The study did not focus on real-time segmentation, and future work is suggested to explore its application in real-time assessment of left ventricle function and structure.
  • Computational Resources: The experiments were conducted on a CPU, and the authors speculate that using a GPU would further accelerate the program.

Summary updated

Abstract

Segmenting the left ventricle from the transgastric short-axis views (TSVs) on transesophageal echocardiography (TEE) is the cornerstone for cardiovascular assessment during perioperative management. Even for seasoned professionals, the procedure remains time-consuming and experience-dependent. The current study aims to evaluate the feasibility of deep learning for automatic segmentation by assessing the validity of different U-Net algorithms. A large dataset containing 1388 TSV acquisitions was retrospectively collected from 451 patients (32% women, average age 53.42 years) who underwent perioperative TEE between July 2015 and October 2023. With image preprocessing and data augmentation, 3336 images were included in the training set, 138 images in the validation set, and 138 images in the test set. Four deep neural networks (U-Net, Attention U-Net, UNet++, and UNeXt) were employed for left ventricle segmentation and compared in terms of the Jaccard similarity coefficient (JSC) and Dice similarity coefficient (DSC) on the test set, as well as the number of network parameters, training time, and inference time. The Attention U-Net and U-Net++ models performed better in terms of JSC (the highest average JSC: 86.02%) and DSC (the highest average DSC: 92.00%), the UNeXt model had the smallest network parameters (1.47 million), and the U-Net model had the least training time (6428.65 s) and inference time for a single image (101.75 ms). The Attention U-Net model outperformed the other three models in challenging cases, including the impaired boundary of left ventricle and the artifact of the papillary muscle. This pioneering exploration demonstrated the feasibility of deep learning for the segmentation of the left ventricle from TSV on TEE, which will facilitate an accelerated and objective alternative of cardiovascular assessment for perioperative management.
BibTeX
@inproceedings{tian2024deep,
  author = {Yuan Tian and Wenting Qin and Zihang Zhao and Chunrong Wang and Yajie Tian and Yuelun Zhang and Kai He and Yuguan Zhang and Le Shen and Zhuhuang Zhou and Chunhua Yu},
  title = {{Deep Learning Based Automatic Left Ventricle Segmentation from the Transgastric Short-Axis View on Transesophageal Echocardiography: A Feasibility Study}},
  booktitle = {Diagnostics},
  year = {2024},
  doi = {10.3390/diagnostics14151655},
}
Figure