Graph matching and deep neural networks based whole heart and great vessel segmentation in congenital heart disease

Zeyang Yao, Wen Xie, Jiawei Zhang, Haiyun Yuan, Meiping Huang, Yiyu Shi, Xiaowei Xu, Jian ZhuangScientific 2023 · DOI

Abstract

Congenital heart disease (CHD) is a leading cause of mortality among birth defects. Due to significant variations in heart and vessel structures, automatic CHD segmentation using CT images remains a challenge. We propose a hybrid framework combining deep learning for regular structures (chambers and myocardium) and graph matching for complex vascular topologies. By extracting connection information and applying graph matching, we determine vessel categories accurately. Experimental results on 68 3D CT images covering 14 CHD types show a 12% increase in Dice score over state-of-the-art methods. Clinical evaluation by specialists confirms the method's effectiveness, paving the way for future clinical use.

Introduction

Congenital Heart Disease (CHD) involves structural defects in the heart and major vessels present from birth. For surgeons, accurate 3D models are indispensable for planning complex repairs. Traditionally, creating these models requires manual segmentation of CT scans—a labor-intensive process taking several hours.

The primary technical hurdle is the extreme topological variability of CHD. Unlike a standard heart, a CHD heart may feature chambers in atypical positions, transposed vessels, or missing structures. While standard Deep Learning models like U-Nets excel at recognizing local textures, they often struggle with these global structural variations, leading to misclassification of vessels based on position rather than connectivity.

Methods

The proposed framework utilizes a multi-stage hybrid pipeline designed to leverage the strengths of both neural networks and graph theory:

  1. Chamber Segmentation: A 3D U-Net segments the four heart chambers and the myocardium, which remain relatively regular even in CHD cases.
  2. Blood Pool Refinement: A 2D U-Net performs high-resolution segmentation of the "blood pool" to sharpen the boundaries of the chambers identified in the first step.
  3. Vessel Topology via Graph Matching: After isolating the vessels from the blood pool, their structures are thinned into 1D skeletons and converted into mathematical graphs.
  4. Classification via Earth Mover’s Distance (EMD): The system compares these patient-specific graphs against a "Graph Library" of known CHD types. By calculating the EMD, the system identifies vessels (e.g., Aorta vs. Pulmonary Artery) based on their connectivity logic rather than spatial coordinates.
The workflow from CT imaging (a) to 3D segmentation (b), and its clinical applications in augmented reality surgical guidance (c) and 3D printing (d).

Experimental Setup

The study utilized a specialized dataset of 68 3D CT Angiography (CTA) images covering 14 different types of CHD. These ranged from "simple" defects (e.g., ASD/VSD) to "complex" malformations like Tetralogy of Fallot. Ground truth labels were manually generated by experienced radiologists, a process requiring significant time per image. The model was benchmarked against Seg-CNN, a leading method for whole-heart segmentation in normal anatomy.

Key Results and Discussion

The hybrid approach demonstrated a significant performance leap. The proposed method achieved an overall Dice score of 78.3%, representing a 12% improvement over the 66.5% achieved by Seg-CNN.

Detailed 3D segmentation of heart chambers and great vessels in a complex CHD case.

The most notable gains were observed in vessel identification. By prioritizing topology over texture, the graph-matching component correctly identified connections that purely pixel-based models often missed. Clinical validation by two cardiovascular specialists using the Van Praagh classification system further supported these findings; the model correctly identified atrioventricular and great vessel relationships in 70 out of 71 cases.

A key insight from this work is the "Hybrid Advantage." While pure deep learning can fail on "out-of-distribution" geometries, the integration of a hard-coded graph library based on medical knowledge provides a "safety rail," ensuring anatomically logical predictions.

Conclusion

While the system shows great promise, some limitations remain. GPU memory constraints necessitated processing down-sampled images in the 3D U-Net, potentially losing fine details. Additionally, the system still faces challenges with highly irregular anomalous veins. Future work will focus on developing an end-to-end trainable system where 2D and 3D networks are fine-tuned together to further enhance precision and clinical utility.

Summary updated

Abstract

Congenital heart disease (CHD) is one of the leading causes of mortality among birth defects, and due to significant variations in the whole heart and great vessel, automatic CHD segmentation using CT images has been always under-researched. Even though some segmentation algorithms have been developed in the literature, none perform very well under the complex structure of CHD. To deal with the challenges, we take advantage of deep learning in processing regular structures and graph algorithms in dealing with large variations and propose a framework combining both the whole heart and great vessel segmentation in complex CHD. We benefit from deep learning in segmenting the four chambers and myocardium based on the blood pool, and then we extract the connection information and apply graph matching to determine the categories of all the vessels. Experimental results on 68 3D CT images covering 14 types of CHD illustrate our framework can increase the Dice score by 12% on average compared with the state-of-the-art whole heart and great vessel segmentation method in normal anatomy. We further introduce two cardiovascular imaging specialists to evaluate our results in the standard of the Van Praagh classification system, and achieves well performance in clinical evaluation. All these results may pave the way for the clinical use of our method in the incoming future.
BibTeX
@inproceedings{yao2023graph,
  author = {Zeyang Yao and Wen Xie and Jiawei Zhang and Haiyun Yuan and Meiping Huang and Yiyu Shi and Xiaowei Xu and Jian Zhuang},
  title = {{Graph matching and deep neural networks based whole heart and great vessel segmentation in congenital heart disease}},
  booktitle = {Scientific},
  year = {2023},
  doi = {10.1038/s41598-023-34013-1},
}
Figure