TL;DR
This paper introduces DE-Net, a novel deep convolutional neural network specifically designed for building extraction from high-resolution remote sensing imagery. DE-Net achieves state-of-the-art performance on benchmark datasets without pre-training, demonstrating the effectiveness of a task-specific architecture over fine-tuning general-purpose models.
What Problem Does This Paper Address?
Accurate building extraction from high-resolution remote sensing (HRRS) imagery is crucial for numerous applications like urban planning, land-use mapping, and change detection. While deep learning has shown promise, existing methods often adapt general-purpose image segmentation networks. These networks can struggle with the unique challenges of HRRS data, such as varying illumination, complex textures, and the need to preserve fine spatial details. Traditional approaches relied on hand-crafted features, which are brittle and sensitive to environmental changes. The core problem is developing a deep learning architecture that is optimized for the specific characteristics of building extraction from HRRS imagery, preserving spatial information effectively throughout the network.
Methods
The authors propose DE-Net, a fully convolutional network with a custom architecture designed to preserve spatial information and enhance building discrimination. Key components include:
- Inception-style Downsampling Modules: These modules combine striding convolutions and max-pooling to capture more information during downsampling while reducing spatial resolution.
- Encoding Modules with SELU: These modules utilize linear residual blocks with the Scaled Exponential Linear Unit (SELU) activation function. SELU is chosen over ReLU to avoid "dead neurons" and preserve valuable information, promoting self-normalization.
- Compressing Modules: These modules reduce feature channels, particularly in the latter half of the network.
- Densely Upsampling Convolution (DUC) Module: This novel upsampling approach reshapes feature maps to directly produce a high-resolution pixel-wise prediction, avoiding traditional upsampling methods like bilinear interpolation or deconvolution that can lose spatial detail.
- Dice and Binary Cross-Entropy Loss: A combined loss function is used to address the class imbalance inherent in building extraction (many background pixels vs. few building pixels), improving the model's ability to learn from minority classes.
DE-Net is designed to be trained from scratch, emphasizing the importance of a task-specific architecture.
Datasets
The paper uses two primary datasets:
- WHU Building Dataset: An open-source dataset containing 8189 tiles of 512x512 pixels with a spatial resolution of 0.3m, covering an area of 450 km² in New Zealand. This dataset is rich in building appearance diversity and label accuracy.
- Suzhou Satellite Building Dataset: A newly established dataset created by the authors from GF-2 multispectral satellite imagery of Suzhou, China. This dataset covers 8488 km² with a spatial resolution of 0.8m and consists of 1184 images. It was created to evaluate performance on datasets with fewer samples and potentially lower annotation accuracy, simulating real-world scenarios where large, well-annotated datasets are scarce.
Key Insights and Ideas
- Task-Specific Architecture Outperforms Generalization: The study strongly suggests that designing a network specifically for building extraction (like DE-Net) and training it from scratch can outperform fine-tuning general-purpose segmentation networks (like SegNet, RefineNet, DeepLab v3+) that were pre-trained on unrelated datasets (e.g., ImageNet). This challenges the conventional wisdom that pre-training is always beneficial.
- Importance of Spatial Information Preservation: The success of DE-Net highlights the critical role of preserving spatial details throughout the network, especially during downsampling and upsampling. The DUC module is a key innovation in this regard.
- SELU for Information Preservation: Using SELU in encoding modules, rather than the more common ReLU, contributes to better information preservation and potentially self-normalization, leading to improved performance.
- Data Quality vs. Model Complexity: On the more challenging Suzhou dataset, DE-Net's robust performance, even with fewer parameters and no pre-training, suggests that model complexity alone isn't the sole determinant of success, especially when data is limited or noisy. A well-designed, task-specific architecture can be more effective.
Key Results
- WHU Dataset: DE-Net achieved state-of-the-art performance on the WHU dataset, outperforming SegNet, U-Net, RefineNet, and DeepLab v3+ in recall, F1-Score, and IoU metrics, without pre-training. It achieved an IoU of 90.12 ± 0.24.
- Suzhou Dataset: DE-Net also significantly outperformed the other evaluated networks on the self-built Suzhou dataset, achieving the highest F1-Score (74.38 ± 1.85) and IoU (74.38 ± 1.85). This dataset presented a more challenging scenario due to lower image quality and fewer samples.
- Generalization Ability: Visual results showed DE-Net producing cleaner and more accurate building extraction, with better recovery of spatial details and robustness to noise like color variations and tree cover. It demonstrated superior generalization ability on unseen samples.
- Efficiency: DE-Net is relatively efficient, with a low parameter count (9.63 million) and competitive training time (81 seconds/epoch) compared to other complex models.
Limitations and Future Work
- Pre-training on Related Datasets: While DE-Net excels without pre-training on unrelated datasets, the authors suggest that for scenarios with insufficient data, pre-training on a related large-scale dataset (like WHU) could still be beneficial. Future work could explore designing new networks and pre-training them on relevant datasets before fine-tuning.
- Further Improvements: The authors suggest integrating techniques like group normalization and generative adversarial training to further enhance DE-Net's performance.
- Broader Applications: Future research could extend DE-Net to classify other land-cover types (water, forest, roads) and apply it to different data sources, such as multispectral or synthetic aperture radar (SAR) data, due to its flexible input band handling. Data fusion is also a promising avenue.
Summary updated