Abstract
Traditional semantic segmentation architectures rely heavily on an encoder-decoder structure, where the decoder is responsible for recovering spatial resolution lost during feature extraction. However, these decoders introduce significant computational overhead. This paper proposes a "decoder-less" architecture utilizing the Depth-to-Space (D2S) operation. By replacing learnable upsampling layers with a zero-parameter pixel rearrangement, the proposed model achieves competitive performance in binary segmentation tasks, such as road extraction, while reducing computational costs by approximately 50%.
Introduction
In computer vision, semantic segmentation typically involves an encoder that extracts high-level features and a decoder that reconstructs the original image resolution. While effective, decoders—using deconvolution or unpooling—often double the memory and FLOP requirements of a network. This research investigates whether complex decoders are truly necessary for relatively simple binary segmentation tasks. The authors hypothesize that spatial information is preserved within deep feature maps and can be recovered through simple rearrangement rather than expensive reconstruction.
Methodology
The core of the proposed architecture is the Depth-to-Space (D2S) operation. Instead of a traditional decoder, the network employs a standard pre-trained backbone (e.g., ResNet50 or VGG16) followed by a final $1 \times 1$ convolution to adjust channel depth. The D2S block then performs a periodic shuffling operation, unfolding the extra depth into spatial dimensions to restore the original resolution.
This approach has zero learnable parameters and negligible computational cost. By shifting the spatial reconstruction burden to the depth dimension, the encoder learns to organize feature maps specifically for the subsequent shuffle, effectively avoiding common upsampling artifacts like checkerboard patterns.
Experimental Results
The models were evaluated on the DeepGlobe Road Extraction dataset, which consists of high-resolution satellite imagery. The task is a challenging binary segmentation problem due to the high class imbalance between thin roads and the vast background.
Quantitative Performance
The D2S-based models significantly outperformed the standard SegNet baseline in terms of Pixel Intersection over Union (IoU):
- ResNet50-D2S: 0.6060 IoU
- VGG16-BN-D2S: 0.5897 IoU
- SegNet: 0.5612 IoU
Beyond accuracy, the D2S approach reduced total computation by nearly half compared to traditional encoder-decoder pairs. While specialized ensembles in the DeepGlobe competition reached higher scores (up to 0.6560 IoU), the D2S model offers a highly efficient single-model alternative with only a marginal performance gap.
Qualitative Analysis
Visual results demonstrate that the decoder-less models effectively capture road networks even in complex satellite imagery. The network successfully learns to store fine-grained spatial details within the channel features of the final encoder layer.
Conclusion
The study demonstrates that for binary segmentation tasks, complex decoders may be redundant. The Depth-to-Space operation provides a computationally efficient alternative that matches or exceeds the performance of traditional architectures. Future work will explore the scalability of this decoder-less approach to multi-class segmentation problems where inter-class boundaries are more complex.
Summary updated