Graph Convolutional Neural Networks for Automated Echocardiography View Recognition: A Holistic Approach

Sarina Thomas, Cristiana Tiago, Børge Solli Andreassen, S.o. Aase, Jurica Šprem, Erik Steen, Anne H. Schistad Solberg, Guy Ben-YosefLecture Notes in Computer Science 2023 · DOI · arXiv · OpenAlex

Introduction

In cardiology, echocardiography (ultrasound) is the primary diagnostic tool. However, obtaining high-quality images that adhere to clinical guidelines—such as the standard "4-chamber view"—is challenging. Conventional deep learning approaches treat view recognition as a simple classification problem. These "black box" models often fail to provide the spatial context necessary to determine if a heart is tilted, foreshortened, or partially occluded, which can render an image useless for clinical measurement. Furthermore, training robust models is hindered by the extreme scarcity of fully annotated 3D cardiac datasets.

This research proposes a "holistic" approach that moves beyond simple labels by reconstructing the 3D pose of the heart from a single 2D slice. By leveraging Graph Convolutional Neural Networks (GCNs) and synthetic data generation, the system provides a richer understanding of image quality and anatomical orientation.

Methodology

The authors developed a multi-stage pipeline that bridges the gap between 3D geometric modeling and 2D clinical imaging:

  1. 3D Mesh Embedding: A template mesh representing the four heart chambers (LV, RV, LA, RA) was used to ensure consistent vertex correspondence across different patient anatomies.
  2. Synthetic Data Generation: To overcome data scarcity, 3D meshes were "sliced" at various angles to create 2D segmentation masks. An adversarial diffusion model then transformed these clean masks into realistic synthetic ultrasound images.
  3. GCN Architecture: The core model employs a ResNet50 encoder to extract features from an ultrasound image. These features are passed to a GCN decoder that utilizes spiral convolutions to learn spatial relationships between mesh vertices, ultimately predicting 3D coordinates that match the input 2D image.
Overview of the proposed pipeline, including synthetic data generation via diffusion models and the GCN architecture for 3D mesh reconstruction.

Experimental Setup

The model was trained and evaluated using two primary datasets:

  • Synthetic Dataset: 4,258 images generated from 20 patient meshes, providing ground truth 3D coordinates for supervised learning.
  • Clinical Dataset: 1,566 real-world clinical images (GE scanner) used to test the model's ability to generalize from synthetic training to real patients.

Key Results and Discussion

The 3DGCN model demonstrated significant improvements over traditional architectures. On clinical test cases, the 3DGCN achieved a weighted average accuracy of 0.75, dramatically outperforming a standard ResNet50 classifier, which reached only 0.46.

Performance evaluation showing confusion matrices for view recognition and Intersection over Union (IoU) distributions for chamber localization across synthetic and clinical cases.

Beyond classification, the model achieved high Mean Intersection over Union (mIoU) scores for chamber localization, such as 0.88 for the Left Ventricle in clinical cases. This suggests that the model inherently understands cardiac anatomy; by treating the heart as a graph of connected points, it ensures that predicted movements of one structure (like the LV) are anatomically logical relative to others.

Limitations and Future Work

A primary challenge remains the domain gap between synthetic and real data. The model occasionally struggled to differentiate between 4-chamber and 5-chamber views because the current 3D meshes do not yet include the Left Ventricular Outflow Tract (LVOT). Additionally, the current iteration focuses on End-Diastole frames. Future research aims to incorporate 4D models (3D + time) to account for the full cardiac cycle and fuse information from multiple views to further refine the 3D reconstruction.

Summary updated

BibTeX
@inproceedings{thomas2023graph,
  author = {Sarina Thomas and Cristiana Tiago and Børge Solli Andreassen and S.O. Aase and Jurica Šprem and Erik Steen and Anne H. Schistad Solberg and Guy Ben-Yosef and Anne Schistad Solberg},
  title = {{Graph Convolutional Neural Networks for Automated Echocardiography View Recognition: A Holistic Approach}},
  booktitle = {Lecture Notes in Computer Science},
  year = {2023},
  doi = {10.1007/978-3-031-44521-7\_5},
}
Figure