Abstract
Retinal vessel segmentation plays an important role in the diagnosis of eye-related diseases and biomarkers discovery. Existing works perform multi-scale feature aggregation in an inter-layer manner, which may result in limited performance, especially on thin vessels. We propose Pyramid-Net, featuring intra-layer pyramid-scale aggregation blocks (IPABs) to fuse multi-scale features within each layer. Enhanced by pyramid inputs, deep supervision, and pyramid skip connections, Pyramid-Net achieves state-of-the-art performance on DRIVE, STARE, and CHASE-DB1 datasets. Experimental results show significant improvements in thin vessel segmentation and a large reduction in computational cost.
Introduction
Retinal vessel segmentation is a critical task for diagnosing systemic diseases like diabetes, hypertension, and glaucoma. While deep learning has made massive strides, "thin vessels" (those only 1–2 pixels wide) remain a major hurdle. Standard Convolutional Neural Networks (CNNs) rely on pooling and downsampling to capture high-level context, but this process often destroys the delicate structural information of small vessels.
Existing "multi-scale" solutions usually perform inter-layer aggregation—fusing features from a low-resolution layer with a high-resolution one. The authors argue this is insufficient because it only considers features at the specific scales of those layers. Pyramid-Net shifts the focus to intra-layer aggregation, ensuring that every stage of the network processes the "big picture" and "fine details" simultaneously.
Methods
The core of the architecture is the Intra-layer Pyramid-scale Aggregation Block (IPAB). Built on the foundation of a ResNet block, the IPAB splits the feature map into three parallel branches:
- Current Scale: The standard processing path.
- Higher Scale: Uses up-sampling to capture finer details.
- Lower Scale: Uses down-sampling to capture broader context.
To maintain efficiency, channel counts are reduced in auxiliary branches. Three additional enhancements boost performance:
- Pyramid Input Enhancement: Resized versions of the original image are fed directly into encoder IPABs to prevent initial information loss.
- Deep Pyramid Supervision: Loss is calculated at multiple scales within the decoder, forcing the network to learn meaningful features at every resolution.
- Pyramid Skip Connections: An evolution of standard U-Net skip connections allowing the decoder to reuse encoder features across multiple scales (from $l-2$ to $l+2$).
Experimental Results
Pyramid-Net was evaluated on three gold-standard datasets: DRIVE (40 images), STARE (20 images with pathological cases), and CHASE-DB1 (28 images with varied illumination).
The model achieved state-of-the-art (SOTA) performance across all benchmarks. On the DRIVE dataset, it reached an Accuracy of 96.26% and an AUC of 98.32%. Notably, thin vessel segmentation improved by 4.73% over the backbone CE-Net model.
Beyond accuracy, Pyramid-Net is highly efficient. By utilizing channel pruning in auxiliary branches, it achieves a 64.7% reduction in FLOPs compared to U-Net++, requiring only 188.15G FLOPs, making it much faster and lighter than competitors like CB-Net (441.62G).
Conclusion
Pyramid-Net addresses the limitations of inter-layer feature aggregation by introducing intra-layer multi-scale processing. While it excels at thin vessel detection, challenges remain for extremely low-contrast vessels in highly pathological images. Future work will focus on improving robustness to thickness bias and enhancing performance in complex clinical scenarios.
Summary updated