Echo-SyncNet: Self-supervised Cardiac View Synchronization in Echocardiography

Fatemeh Taheri Dezaki, Christina Luong, Tom Ginsberg, Robert Rohling, Ken Gin, Purang Abolmaesumi, Teresa Tsang

Introduction

Accurate cardiac assessment often requires the integration of multiple echocardiographic views to capture the heart's complex three-dimensional motion. Traditionally, synchronizing these views—ensuring that frames from different angles represent the same physiological moment—relies on electrocardiogram (ECG) gating. However, in many clinical scenarios, such as emergency departments or point-of-care ultrasound (POCUS) in ambulances, ECG leads are impractical or unavailable. Echo-SyncNet addresses this gap by providing a self-supervised framework for ECG-free temporal synchronization of cardiac ultrasound videos.

The Synchronization Challenge

In clinical cardiology, metrics like Ejection Fraction (EF) are calculated by comparing the heart's volume at End-Diastole (full expansion) and End-Systole (maximum contraction). To do this accurately across different views, such as the Apical 4-chamber (AP4) and Apical 2-chamber (AP2) views, the videos must be perfectly aligned.

Manual synchronization is labor-intensive and error-prone due to varying heart rates and the distinct structural appearances of different views. Echo-SyncNet aims to learn the intrinsic rhythm of the heart directly from the video data, creating a "universal clock" that functions across patients and camera angles without external hardware.

Methodology: Self-Supervised Learning for Cardiac Phase

Echo-SyncNet utilizes a self-supervised learning (SSL) approach, extracting features from raw video without human-labeled ground truth. The architecture employs a ResNet-50 backbone for spatial feature extraction, followed by 3D convolutional layers to capture temporal dynamics. Each frame is mapped into a 128-dimensional embedding space.

The model is trained using a combination of three loss functions:

  1. Temporal Intra-view Supervision: The model learns the directionality of time by determining the correct chronological order of frame sequences.
  2. Spatial Intra-view Supervision: An n-tuplet loss ensures that frames close in time have similar embeddings, while those further apart in the cardiac cycle are pushed away in the embedding space.
  3. Inter-view Supervision: This core innovation enforces "cycle consistency." If a frame in View A is the nearest neighbor to a frame in View B, the reverse must also hold true. This forces the model to learn a view-invariant representation of the cardiac phase.

Post-training, synchronization is achieved by mapping frames from two different videos into the embedding space and applying Dynamic Time Warping (DTW) to find the optimal temporal alignment.

Experimental Results and Clinical Significance

The model was validated using a large internal dataset from Vancouver General Hospital (over 3,000 patients) and the public EchoNet-Dynamic dataset (over 10,000 AP4 echocardiograms).

High-Precision Alignment

Echo-SyncNet achieved a Kendall’s Tau of 0.921, indicating an exceptionally high correlation in temporal ordering between synchronized clips. This suggests the model effectively captures the underlying physiological rhythm.

Clinical Utility and Keyframe Detection

In one-shot keyframe detection tasks—identifying End-Diastole and End-Systole—the model's error was limited to approximately 3 to 4 frames. This performance is significant as it falls within the typical range of inter-observer variability among professional sonographers, suggesting the model's potential for clinical decision support.

Generalization and Robustness

A key finding was the model's ability to generalize to unseen views. Although primarily trained on AP2 and AP4 pairs, it successfully synchronized other views like the Parasternal Long Axis (PLAX) and Apical 5-chamber (AP5). Furthermore, the SSL approach proved more robust than supervised models in low-data regimes, as it harvests more nuanced information from the raw video than simple phase labels provide.

Conclusion and Future Directions

Echo-SyncNet demonstrates that the intrinsic motion of the heart provides a sufficient signal for high-precision synchronization without the need for ECG. While current limitations include the requirement for at least one full cardiac cycle, the framework opens new possibilities for real-time diagnostic feedback in POCUS devices. Future research will explore the detection of arrhythmias and other cardiac abnormalities directly within the learned embedding space.

Summary updated

BibTeX
@article{dezaki2021echo,
  author = {Fatemeh Taheri Dezaki and Christina Luong and Tom Ginsberg and Robert Rohling and Ken Gin and Purang Abolmaesumi and Teresa Tsang},
  title = {{Echo-SyncNet: Self-supervised Cardiac View Synchronization in Echocardiography}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2021},
  doi = {10.1109/tmi.2021.3071951},
}
Figure