TL;DR
This paper introduces a novel semi-supervised learning framework for detecting buildings in high-resolution remote sensing images. By leveraging both limited labeled data and abundant unlabeled data, their method significantly improves building detection accuracy, addressing the costly and time-consuming nature of manual annotation.
What Problem Does This Paper Address?
Accurate building information is crucial for urban planning, sustainability, and disaster management. High-resolution remote sensing images (HR RSIs) offer detailed building information, but detecting buildings from them is challenging due to complex backgrounds, diverse appearances, and occlusions. Traditional methods often rely on handcrafted features, which are laborious and lack generalization. While deep learning methods have shown promise, they typically require a massive amount of meticulously labeled data, which is expensive and time-consuming to acquire for RSIs. This paper tackles the problem of building detection in HR RSIs by developing a semi-supervised approach that reduces the reliance on extensive manual labeling.
Methods
The proposed framework is built upon a teacher-student mutual learning paradigm and consists of two core modules:
- Color and Gaussian Augmentation (CGA) Module: This module aims to enhance the training of an anchor-free FCOS (Fully Convolutional One-Stage) object detector. It enriches the diversity of building features and effectively increases the quantity of labeled data by applying color augmentations (adjusting saturation, brightness, contrast, sharpness) and Gaussian noise to the existing labeled images. This helps the detector learn more robust features.
- Consistency Learning (CL) Module: This module leverages unlabeled data to improve detection performance. It introduces a novel consistency loss that encourages the teacher and student models to produce consistent predictions on different augmented versions of the same unlabeled image. This consistency regularization helps the model generalize better to unseen data and enhances its detection ability on unlabeled RSIs.
The framework combines a supervised loss (for labeled data) with this unsupervised consistency loss (for unlabeled data) to train the student model. The teacher model is then refined using Exponential Moving Average (EMA) of the student model's weights, which is a common technique in semi-supervised learning to stabilize training and improve performance.
Datasets
The authors evaluated their framework on three standard building detection datasets:
- WHU Aerial Imagery Dataset: Contains aerial imagery from Christchurch and New Zealand with high spatial resolution.
- CrowdAI Building Dataset: Composed of satellite images with challenges like shadows and occlusions.
- TCC Building Dataset: Features images sampled from Google Maps, including non-orthophoto images and dense small buildings.
For training, they created mixed datasets by randomly selecting 1% of the labeled images from the training sets as labeled data and using the remaining training images as unlabeled data.
Key Insights and Ideas
- Semi-supervised learning is highly effective for building detection in HR RSIs: The paper demonstrates that by effectively utilizing unlabeled data, significant performance gains can be achieved, alleviating the bottleneck of manual annotation.
- Data augmentation is crucial for anchor-free detectors: The CGA module highlights the importance of enriching feature diversity, especially for anchor-free models like FCOS, which are less sensitive to predefined anchor boxes.
- Consistency learning bridges the gap between labeled and unlabeled data: The CL module's ability to enforce consistent predictions across perturbed unlabeled images is key to improving the model's generalization and robustness.
- Teacher-student mutual learning provides a stable training paradigm: The EMA update strategy for the teacher model helps to consolidate learned knowledge and improve overall performance.
- Anchor-free detectors are well-suited for diverse building shapes: The choice of FCOS as the base detector is justified by the diverse shapes and sizes of buildings in RSIs, where anchor-based methods might struggle.
Key Results
The proposed semi-supervised framework achieved state-of-the-art performance on all three tested datasets.
- WHU Aerial Imagery Dataset: Achieved an AP of 0.380 and AP50 of 0.736, outperforming both anchor-based and anchor-free supervised methods, as well as existing semi-supervised methods.
- CrowdAI Building Dataset: Achieved an AP of 0.365 and AP50 of 0.704, showing significant improvements over supervised baselines.
- TCC Building Dataset: Achieved an AP of 0.133 and AP50 of 0.370, demonstrating robustness in challenging scenarios.
Qualitative results showed that the proposed framework could better detect buildings in dense areas, distinguish buildings from similar-colored roads, and handle occlusions and complex shapes more effectively than baseline methods.
Limitations and Future Work
- Performance with sufficient labeled data: The authors note that their method is not superior to supervised methods when a relatively large amount of labeled data is available. This suggests that further investigation is needed to optimize performance in such scenarios.
- Introduction of additional features: The paper suggests that incorporating features like building center-ness could further improve building detection from HR RSIs.
- Generalization across diverse regions: While the study shows good generalization across different regions, the performance was slightly lower in the Serbian region due to sparse building distribution. Future work could explore strategies to better handle such sparse scenarios.
Summary updated