Abstract
This paper investigates the synergy between "long" skip connections, which recover spatial information, and "short" skip connections, which facilitate gradient flow, to enable the training of extremely deep Fully Convolutional Networks (FCNs). By hybridizing the architectures of U-Net and ResNet, the authors demonstrate that deep models can achieve near state-of-the-art results in biomedical image segmentation using significantly fewer parameters and no complex post-processing.
Introduction
Biomedical image segmentation requires high-precision pixel-level labeling. Standard Convolutional Neural Networks (CNNs) typically utilize a "contracting" path that downsamples images to capture context; however, this process often discards fine-grained spatial information necessary for accurate boundary detection.
While Fully Convolutional Networks (FCNs) like the U-Net introduced "long" skip connections to recover this lost data, they still face the vanishing gradient problem. As networks grow deeper (reaching hundreds of layers), the training signal weakens as it propagates backward, making deep FCNs difficult to optimize. This research explores whether adding "short" skip connections—residual connections—can unlock the potential of very deep architectures for medical imaging.
Methodology
The proposed architecture integrates two distinct types of skip connections to balance spatial recovery with training stability:
- Long Skip Connections: These link layers from the contracting (encoder) path directly to the expanding (decoder) path at corresponding resolutions, helping the model retain spatial details lost during downsampling.
- Short Skip Connections: Internal to the residual blocks, these allow gradients to bypass non-linearities, creating a "highway" for the gradient to flow uninterrupted to the earliest layers.
The architecture employs three block types: bottleneck, basic, and simple. To maintain efficiency, $1 \times 1$ convolutions adjust filter counts, while simple decimation/repetition handles spatial adjustments. The study also compares standard Binary Cross-Entropy (BCE) loss with Dice Loss, the latter being more robust to the class imbalances typical in medical datasets.
Experimental Results
The model was evaluated on the EM ISBI 2012 dataset, consisting of electron microscopy images of the Drosophila larva ventral nerve cord. Despite having only 30 training images, the authors used extensive data augmentation (flipping, shearing, rotations, and spline warping) to improve generalization.
The hybrid architecture achieved competitive scores on the EM challenge metrics:
- Maximal foreground-restricted Rand score ($V_{rand}$): 0.969
- Information theoretic score ($V_{info}$): 0.986
Notably, these results were achieved with only 11 million parameters—a third of the U-Net's 33 million—and without any post-processing (such as watershed transforms or conditional random fields) often required by other top-tier entries.
Key Insights and Discussion
- The "Middle" Problem: Empirical evidence suggested that while long skip connections assist the shallowest and deepest parts of the network, they do not alleviate gradient issues in the middle layers. Short skip connections are essential for ensuring these central layers learn effectively.
- Convergence and Stability: Models utilizing both skip connection types converged significantly faster and exhibited more stable updates during training.
- Stochastic Inference: The use of test-time dropout (averaging multiple passes with dropout enabled) was found to further improve boundary continuity and prediction "cleanliness."
Conclusion
By combining long and short skip connections, the authors proved that architectural depth and clever gradient management can outperform raw model width. While the model does not yet surpass the absolute top of the leaderboard—which often relies on heavy ensembles and post-processing—it provides a highly efficient and powerful baseline for deep biomedical image segmentation. Future work will focus on extending these 2D architectures to 3D medical datasets.
Summary updated