TL;DR
This paper introduces an improved Mask R-CNN model for automatically extracting building footprints from complex Google Earth imagery. By incorporating rotated anchors and a novel Receptive Field Block (RFB) module, the method achieves state-of-the-art performance in detecting and segmenting buildings, even in challenging scenarios with dense arrangements and varied backgrounds.
What Problem Does This Paper Address?
Accurate building footprint extraction from high-resolution remote sensing imagery is crucial for disaster assessment and urban planning. Traditional methods often rely on manual intervention or struggle with complex backgrounds, leading to low accuracy and efficiency. Existing deep learning approaches, while powerful, can face challenges with closely adjacent buildings, edge confusion, and variations in building scale and orientation. This paper aims to bridge this gap by developing a robust deep learning framework capable of handling these complexities.
Methods
The core of the proposed method is an adaptation of the Mask R-CNN architecture. The key innovations include:
- Rotation Anchors: Instead of standard horizontal anchors, the authors introduce rotation anchors in the Region Proposal Network (RPN) stage. This allows the network to directly predict rotated bounding boxes, better capturing the orientation of buildings and reducing background noise.
- Receptive Field Block (RFB): To enhance the segmentation branch's ability to handle multi-scale buildings and complex features, the authors integrate RFB modules. These modules combine atrous convolutions with different rates and kernel sizes, inspired by the human visual system, to capture richer contextual information.
- Leveling ROIs: After the RPN proposes rotated regions of interest (ROIs), these ROIs are rotated anticlockwise to a horizontal orientation. This "leveling" step, followed by ROI Align, standardizes the input to the subsequent multi-branch prediction network.
- Inverse Rotation: Finally, the predicted bounding boxes and masks are rotated back clockwise by the angle predicted by the RPN to obtain the final instance segmentation results.
Datasets
The primary dataset used for training and evaluation is a newly collected large-scale dataset of Google Earth images from Fujian province, China. This dataset comprises 86 typical regions with a spatial resolution of 0.26 m, with images ranging from 1000x1000 to 10,000x10,000 pixels. After manual annotation of building outlines using ArcGIS, the images were cropped into 500x500 patches, resulting in approximately 2000 images and 84,366 building instances. The dataset was split equally for training and testing.
Key Insights and Ideas
- Rotated Bounding Boxes are Crucial: The paper strongly suggests that explicitly modeling the rotation of buildings is key to improving detection accuracy, especially in dense urban environments where buildings are not axis-aligned.
- RFB Modules Boost Feature Representation: The integration of RFB modules effectively enhances the segmentation branch by allowing it to learn more discriminative features across different scales, which is critical for complex remote sensing imagery.
- Instance Segmentation for Dense Buildings: Mask R-CNN's instance segmentation approach is well-suited for separating individual buildings, even when they are closely packed or share boundaries, a common challenge in urban areas.
- Dataset Diversity Matters: The use of a diverse dataset covering urban, town, and village areas with various building types (apartments, villas, rural housing) is essential for training a robust model.
Key Results
The proposed method achieved state-of-the-art performance on the Fujian dataset.
- Detection Task: The proposed method achieved a mean Average Precision (mAP) of 0.9063, outperforming Faster R-CNN-VGG (0.6958), Faster R-CNN-ResNet101 (0.8976), and Mask R-CNN-ResNet101 (0.8996) in the detection task.
- Segmentation Task: In terms of segmentation, the proposed method achieved a precision of 0.8720, recall of 0.9041, and F₁-score of 0.8878, surpassing Mask R-CNN-ResNet101 (precision: 0.8627, recall: 0.8951, F₁-score: 0.8786).
- Qualitative results demonstrated the method's ability to successfully segment buildings that were missed by Mask R-CNN, particularly in challenging scenarios with adjacent buildings and complex backgrounds.
Limitations and Future Work
The authors acknowledge several limitations and suggest future research directions:
- More Diverse Samples: Deep learning models benefit from larger and more diverse datasets. Future work could involve collecting and annotating more building samples, especially those under complex backgrounds and with varied appearances.
- Refining Rotated Angle: The current method regresses the rotated angle in the RPN stage. Future work could explore refining this angle in the second stage, similar to bounding box regression, or by transforming vertical ROIs to infer the angle more accurately.
- Network Compression: The Mask R-CNN framework is computationally intensive. Research into network compression techniques (e.g., lighter backbones, pruning, kernel sparsity) is suggested to improve inference speed and enable deployment on resource-constrained devices.
- Detecting Specific Building Types: Future work could focus on designing networks specifically for detecting closely located small buildings, large-scale structures, and buildings with unusual shapes, as well as handling even more confusing backgrounds.
Summary updated