Introduction
Medical image segmentation is a cornerstone of computer-aided diagnosis. While state-of-the-art deep learning models like U-Net achieve high pixel-level accuracy, they often fail to maintain "anatomical plausibility." Because standard loss functions treat pixels as largely independent, models can produce fragmented organs or holes—results that are biologically impossible. This paper introduces HybridGNet, an architecture designed to enforce structural constraints by combining Convolutional Neural Networks (CNNs) with Graph Neural Networks (GNNs).
Methodology: The HybridGNet Architecture
HybridGNet moves away from pixel-level masks, instead representing anatomical structures as graphs where nodes correspond to landmarks. The architecture follows an encoder-decoder structure:
- CNN Encoder: A residual CNN extracts high-level features from the input image.
- GCNN Decoder: A Graph Convolutional Neural Network (GCNN) uses spectral convolutions to learn spatial relationships between landmarks.
- Image-to-Graph Skip Connections (IGSC): This novel component bridges the gap between image features and graph nodes. Using RoIAlign, the model samples local image features based on current node positions, allowing the decoder to refine landmark locations using local texture.
Experimental Setup
The model was evaluated on several chest X-ray (CXR) datasets:
- JSRT: Primary training set with 120 expert landmarks for heart and lungs.
- Montgomery & Shenzhen: External test sets for domain shift evaluation.
- Padchest: Used for testing robustness against clinical occlusions (e.g., pacemakers) and cardiothoracic ratio (CTR) estimation.
Results and Discussion
HybridGNet demonstrates significant improvements in both quantitative metrics and qualitative plausibility. Unlike pixel-based methods, the graph-based output ensures a continuous, single-component topology by construction.
Key Findings
- Robustness to Occlusions: In cases where images contain "black boxes" or pacemakers, HybridGNet "hallucinates" missing boundaries by relying on its learned anatomical prior, whereas pixel-based models like nnU-Net often produce holes.
- Efficiency: HybridGNet is computationally efficient, with an inference time of 0.53s per image, roughly five times faster than nnU-Net.
- Demographic Bias: A critical observation was the performance drop in pediatric cases. Since the training data lacked children, the model struggled with their different anatomical proportions, highlighting the need for age-diverse training sets.
Conclusion
HybridGNet successfully integrates global anatomical priors with local image features to produce highly plausible segmentations. By guaranteeing topological correctness and offering superior robustness to occlusions, it provides a more reliable tool for clinical tasks like CTR estimation. Future work will explore extending this framework to 3D medical imaging and addressing demographic gaps in training data.
Summary updated