Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixels

Ching-Hsun Tseng, S. C. Chien, Po-Shen Wang, Shin‐Jye Lee, Wei-Huan Hu, Bin Pu, Xiao‐Jun Zeng

Abstract

Motion mode (M-mode) recording is an essential part of echocardiography to measure cardiac dimension and function. However, the current diagnosis cannot build an automatic scheme due to three fundamental obstructions: the lack of open datasets to bridge M-mode echocardiography with real-time instance segmentation (RIS), the time-consuming nature of manual labeling, and the limited receptive fields in existing backbones (e.g., ResNet) which are inefficient at covering the large spatial extent of cardiac structures. We propose RAMEM, a real-time automatic M-mode echocardiography measurement scheme. RAMEM contributes three key components: 1) MEIS, a new dataset for M-mode instance segmentation; 2) Panel Attention, a local-to-global efficient attention mechanism using pixel-unshuffling; and 3) AMEM, an efficient algorithm for automatic clinical measurement. Experimental results show that RAMEM surpasses existing RIS backbones on PASCAL 2012 SBD and outperforms human experts in real-time clinical testing.

Introduction

M-mode echocardiography provides a high-resolution temporal view of cardiac structures, making it indispensable for assessing ventricular wall thickness and valve movement. Despite its clinical importance, the measurement process remains largely manual. Clinicians must identify specific "anchors" on scrolling ultrasound images to calculate indices, a process that is not only slow but also subject to significant inter-observer variability. Automating this process requires bridging M-mode imaging with Real-time Instance Segmentation (RIS), yet standard neural networks face a significant "receptive field" bottleneck when dealing with large objects that span significant portions of the image.

The RAMEM Framework

The proposed Real-time Automatic M-mode Echocardiography Measurement (RAMEM) framework seeks to replace the traditional manual pipeline with a fully automated, end-to-end system.

The proposed RAMEM framework compared to the traditional manual pipeline.

The framework consists of three integrated components: the MEIS dataset for training, an updated UPANets V2 backbone featuring Panel Attention, and the AMEM algorithm for post-processing segmentation masks into clinical metrics. By automating the labeling process, RAMEM ensures consistent results and significantly reduces the diagnostic time required for echocardiographic examinations.

Panel Attention: Overcoming the Receptive Field Bottleneck

Cardiac structures in M-mode images often occupy a large percentage of the frame. Standard CNNs rely on local convolutions that require many layers to achieve a global view. While Non-Local (NL) attention can provide this global context, its $O(N^2)$ complexity typically prevents real-time performance.

Panel Attention solves this by utilizing "pixel-unshuffling." This operation is a lossless downsampling technique that reshapes spatial information into the channel dimension. By reducing the spatial resolution while preserving data, the model can apply grouped convolutions in a "panel" pattern. This approach mimics the benefits of dilated convolutions and global attention but maintains the efficiency required for real-time inference.

Comparison of attention mechanisms: (a) NL Attention before down-sampling, (b) NL Attention after down-sampling, and (c) the proposed Panel Attention which efficiently covers global pixels.

The MEIS Dataset

To facilitate the development of RIS for echocardiography, we introduced the MEIS (M-mode Echocardiograms for Instance Segmentation) dataset. It contains 2,639 images across two primary clinical views: the Aortic Valve (AV) and the Left Ventricle (LV). Each image features expert-verified masks for critical structures, such as the interventricular septum (IVS) and the left ventricular posterior wall (LVPW), providing a robust benchmark for automated systems.

Experimental Results

RAMEM was evaluated against both standard computer vision benchmarks and human performance. On the PASCAL 2012 SBD benchmark, the Panel Attention-equipped backbone achieved an Average mAP of 42.69, significantly outperforming the baseline YOLACT (35.73).

In clinical testing on the MEIS dataset, RAMEM demonstrated superior accuracy compared to human experts. For Left Ventricle measurements, the system achieved a Mean Absolute Error (MAE) of 0.144, whereas human candidates averaged an MAE of 0.425. Crucially, the system maintains a processing speed of approximately 24.45 FPS, enabling real-time integration into clinical diagnostic workflows.

Conclusion

RAMEM addresses the long-standing challenges of automating M-mode echocardiography. By combining a novel, efficient attention mechanism with a specialized dataset and measurement algorithm, the framework provides a path toward faster, more consistent, and more accurate cardiac diagnostics. The code and dataset are available at https://github.com/hanktseng131415go/RAME.

Summary updated

Abstract

Motion mode (M-mode) recording is an essential part of echocardiography to measure cardiac dimension and function. However, the current diagnosis cannot build an automatic scheme, as there are three fundamental obstructs: Firstly, there is no open dataset available to build the automation for ensuring constant results and bridging M-mode echocardiography with real-time instance segmentation (RIS); Secondly, the examination is involving the time-consuming manual labelling upon M-mode echocardiograms; Thirdly, as objects in echocardiograms occupy a significant portion of pixels, the limited receptive field in existing backbones (e.g., ResNet) composed from multiple convolution layers are inefficient to cover the period of a valve movement. Existing non-local attentions (NL) compromise being unable real-time with a high computation overhead or losing information from a simplified version of the non-local block. Therefore, we proposed RAMEM, a real-time automatic M-mode echocardiography measurement scheme, contributes three aspects to answer the problems: 1) provide MEIS, a dataset of M-mode echocardiograms for instance segmentation, to enable consistent results and support the development of an automatic scheme; 2) propose panel attention, local-to-global efficient attention by pixel-unshuffling, embedding with updated UPANets V2 in a RIS scheme toward big object detection with global receptive field; 3) develop and implement AMEM, an efficient algorithm of automatic M-mode echocardiography measurement enabling fast and accurate automatic labelling among diagnosis. The experimental results show that RAMEM surpasses existing RIS backbones (with non-local attention) in PASCAL 2012 SBD and human performances in real-time MEIS tested. The code of MEIS and dataset are available at https://github.com/hanktseng131415go/RAME.
BibTeX
@misc{tseng2023real,
  author = {Ching-Hsun Tseng and S. C. Chien and Po-Shen Wang and Shin‐Jye Lee and Wei-Huan Hu and Bin Pu and Xiao‐Jun Zeng},
  title = {{Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixels}},
  year = {2023},
  eprint = {2308.07717},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.07717},
}
Figure