Abstract
Rapid building damage assessment is critical for effective disaster response, yet traditional deep learning models are hindered by a heavy reliance on large-scale, human-labeled datasets. This paper presents a self-supervised learning (SSL) framework designed to bridge the label gap in disaster scenarios. By integrating global contrastive learning with local image reconstruction in an asymmetric twin network, the proposed model learns robust features from unlabeled satellite imagery. Experimental results on the xBD benchmark demonstrate that this approach significantly outperforms standard ImageNet pre-training, particularly in data-scarce environments, achieving an F1-score of 0.390 with only 1% of available labels.
Introduction
When natural disasters such as hurricanes, earthquakes, or floods occur, emergency responders require immediate, actionable intelligence to prioritize rescue and recovery efforts. Satellite imagery provides a high-level view of affected regions, but the manual assessment of building damage is time-consuming and labor-intensive. While deep learning models offer a path toward automation, they typically suffer from a "data hunger" problem, requiring thousands of expertly labeled pre- and post-disaster image pairs.
In the immediate aftermath of a new disaster, such labels are rarely available. Furthermore, existing models often rely on pre-training from generic datasets like ImageNet, which lack the specific "top-down" perspective and semantic nuances of remote sensing data. This research addresses these challenges by developing a framework capable of learning from unlabeled satellite data, thereby reducing the dependency on manual annotations.
Methodology
The researchers propose a two-stage framework that shifts away from traditional supervised paradigms toward a more flexible, self-supervised approach.
Stage 1: Self-Supervised Pre-training
The core of the framework is an asymmetric twin network architecture designed to learn features from unlabeled images through two parallel tasks:
- Global Contrastive Learning: Utilizing the DINO (Self-Distillation with No Labels) framework, the model compares different augmented views of the same image. A "student" network is trained to predict the output of a "teacher" network. To ensure the model learns meaningful representations rather than trivial solutions, the authors employ sharpening and centering techniques.
- Local Image Reconstruction: Recognizing that satellite imagery contains critical small-scale details, the authors integrated a lightweight decoder. This forces the encoder to reconstruct the original image from its internal features, ensuring the model captures local semantic information, such as building edges and structural textures.
Stage 2: Downstream Fine-tuning
The pre-trained encoder is subsequently integrated into a Siamese Network. This architecture features two identical branches with shared weights to process pre-disaster and post-disaster images simultaneously. The features from both branches are fused and passed to a UPerNet semantic segmentation head. The model produces two primary outputs: a localization mask identifying building footprints and a classification output categorizing damage on a four-level scale (from "no damage" to "destroyed").
Dataset and Experimental Setup
The study utilizes the xBD dataset, the largest and most diverse benchmark for building damage assessment. The dataset comprises 22,068 high-resolution images covering 5,000 km² and includes 19 different disaster types across 15 countries. Annotations include polygons for building footprints and standardized damage levels.
To evaluate the model's efficiency, the researchers compared their SSL approach against standard ImageNet pre-training and other state-of-the-art self-supervised methods, such as MoCo v3 and standard DINO, across varying levels of label availability.
Results and Discussion
The proposed model demonstrated superior performance and robustness, particularly when training data was limited:
- The 1% Label Challenge: When fine-tuned with only 1% of the xBD labels, the model achieved an F1-score of 0.390. This significantly outperformed ImageNet pre-training (0.321) and standard DINO (0.366), proving that domain-specific SSL is more effective than generic pre-training.
- Scaling Performance: With 20% of the labels, the model reached an F1-score of 0.636, maintaining a competitive edge over other initialization methods.
- Feature Visualization: Attention matrix visualizations revealed that the self-supervised encoder naturally learns to focus on building structures without any human guidance. The resulting attention maps closely align with ground-truth building masks.
- Convergence and Efficiency: Training curves indicated that the proposed model converges faster and reaches a higher performance plateau than models initialized with random weights or ImageNet features.
Conclusion and Future Work
This research demonstrates that self-supervised learning can effectively mitigate the data scarcity problem in disaster response. By combining global and local feature learning, the model captures the intricate details necessary for accurate damage assessment from satellite imagery.
Future research directions include exploring deeper transformer architectures to capture more complex patterns and incorporating multi-temporal image sequences to monitor the progression of damage and recovery over time. Additionally, addressing the inherent class imbalance in disaster data—where "destroyed" buildings are significantly rarer than "undamaged" ones—remains a critical area for further refinement.
Summary updated