Abstract
In this work, we address the problem of landmark-based segmentation for anatomical structures. We propose HybridGNet, an encoder-decoder neural architecture which combines standard convolutions for image feature encoding with graph convolutional neural networks to decode plausible representations of anatomical structures. We benchmark the proposed architecture against other standard landmark and pixel-based models for anatomical segmentation in chest X-ray images and find that HybridGNet is more robust to image occlusions. We also show that it can be used to construct landmark-based segmentations from pixel-level annotations. Our experimental results suggest that HybridGNet produces accurate and anatomically plausible landmark-based segmentations by naturally incorporating shape constraints within the decoding process via spectral convolutions.
Introduction
In medical imaging, segmentation typically involves assigning labels to every pixel in an image. While standard architectures like the U-Net excel at this task, they often lack an inherent understanding of global anatomical structure. Consequently, pixel-based models may predict anatomically impossible shapes, such as organs with holes or jagged edges, because they lack topological constraints.
Anatomical structures like lungs, hearts, and bones follow regular, predictable patterns. Traditionally, researchers used Point Distribution Models (PDMs) to capture this, but these models often rely on linear assumptions (like PCA) that struggle with complex variations. HybridGNet bridges this gap by combining the feature-learning power of deep learning with the structural constraints of graph-based representations, ensuring that the output is always a topologically "legal" anatomical shape.
Methods: The HybridGNet Architecture
HybridGNet is an encoder-decoder architecture that operates across two domains: image features and graph representations.
- The Image Encoder: A standard CNN encoder using residual blocks processes a chest X-ray to produce a low-dimensional latent vector representing the image's essential features.
- The Graph Decoder: Instead of a deconvolutional decoder that generates pixels, HybridGNet utilizes a Graph Convolutional Neural Network (GCNN). This decoder takes the latent vector and predicts the spatial coordinates of a set of landmarks (nodes) connected in a predefined graph.
Technical Innovations
- Spectral Graph Convolutions: The model employs Chebyshev convolutions, allowing it to perform operations on the graph using the graph Laplacian. This enables the network to learn spatial relationships between neighboring landmarks.
- VAE Pre-training: To stabilize training, the authors first trained separate Variational Autoencoders (VAEs) for images and landmark shapes, then "stitched" the image encoder to the graph decoder for fine-tuning.
- Dual-Branch Variants: The study also explored "Dual" versions that output both a graph and a standard pixel mask, utilizing skip connections to share information between tasks.
Experimental Setup and Results
The model was evaluated using the JSRT (Japanese Society of Radiological Technology) Database, containing 247 high-resolution chest X-rays with 166 landmarks per image. The dataset was split into training (70%), validation (10%), and testing (20%) sets using 3-fold cross-validation.
Quantitative Performance
HybridGNet outperformed standard baselines, including PCA-based models and fully connected VAEs, across key metrics. Specifically, it achieved lower Mean Squared Error (MSE) and Hausdorff Distance (HD), indicating that its predicted contours were closer to the ground truth.
Key Insights
- Topology as a Prior: By using a graph with fixed connectivity, the model cannot produce a topologically broken shape, naturally incorporating anatomical constraints.
- Robustness to Occlusion: HybridGNet demonstrated superior stability during "stress tests" where portions of the X-ray were obscured. Because it maps the entire image to a global latent space, it can infer missing boundaries based on the surrounding visible anatomy.
- Anatomical Plausibility: Qualitative analysis showed that HybridGNet produced smoother and more realistic shapes than baseline models, particularly for complex structures like the clavicles.
Conclusion
HybridGNet represents a significant step forward in anatomical segmentation by integrating graph-based priors into deep learning architectures. By treating anatomy as a graph rather than a collection of independent pixels, the model achieves greater robustness and plausibility. Future work may explore dynamic graph connectivity and extending the architecture to 3D medical imaging volumes.
Summary updated