AI for Disaster Rapid Damage Assessment from Microblogs

Muhammad Imran, Umair Qazi, Ferda Ofli, Steve Peterson, Firoj AlamAAAI 2022 · DOI

Introduction

In the immediate aftermath of a natural disaster, the first 72 hours represent a critical window for humanitarian response. Effective resource deployment requires precise, real-time information regarding infrastructure damage. Traditional assessment methods, such as ground surveys by experts or satellite imagery, often face significant delays due to hazardous conditions or cloud cover. This paper presents a Rapid Damage Assessment (RDA) system that leverages the high volume of imagery shared on social media platforms like Twitter to provide actionable intelligence for emergency responders.

Problem Statement: The Twitter "Firehose"

While social media provides a wealth of "citizen science" imagery, the primary challenge is information overload. During major events, millions of posts are generated, but the vast majority consist of irrelevant content, such as memes, advertisements, or redundant retweets. For Community Emergency Response Teams (CERTs), manual filtering is prohibitively slow. The RDA system aims to automate the extraction of relevant, unique, and classified damage imagery from this noisy data stream.

Methodology

The RDA system employs a multi-stage computer vision pipeline designed for real-time operation:

  1. Data Collection: The system tracks disaster-specific keywords and geographical bounding boxes via the Twitter API.
  2. Two-Stage De-duplication: To handle the high volume of retweets and near-duplicates, the system first performs URL-based hashing. Subsequently, a fine-tuned VGG16 network extracts deep features to identify and discard visually similar images based on Euclidean distance.
  3. Junk Filtering: A binary classifier removes non-disaster content, achieving a 98% F1-score.
  4. Damage Severity Assessment: Images are classified into three categories: Severe Damage, Mild Damage, and None.
  5. Human-in-the-Loop (HITL): A critical component of the RDA system is the integration of expert feedback. During deployment, responders use a custom interface to verify and correct AI predictions. These corrections are used to retrain the models in real-time, addressing the "domain shift" inherent in different disaster environments.
The RDA annotation interface used by experts to provide real-time feedback for model retraining.

Dataset and Deployment

The system was evaluated using data from Hurricane Dorian (2019). A dataset of 28,050 images was collected and annotated by domain experts. Analysis revealed a significant class imbalance: only 10% of unique images showed actual damage (4% severe, 6% mild). This imbalance underscores the necessity of automated filtering to prevent cognitive overload for human analysts.

An example of an image classified as "Severe Damage," showing substantial structural destruction.

Results and Discussion

The research highlights the impact of domain-specific retraining. A baseline model (Model 1) trained only on historical disaster data achieved a Macro-F1 score of 0.570 when applied to Hurricane Dorian. In contrast, Model 4, which integrated expert-labeled data from the current event along with data augmentation, improved the Macro-F1 score to 0.718.

Key findings include:

  • The 90% Noise Rule: Approximately 90% of social media imagery during a disaster is irrelevant to damage assessment.
  • Domain Shift: Models trained on past disasters generalize poorly to new events due to variations in architecture and environment.
  • Expert Feedback: Real-time retraining using expert labels is more effective than standard data augmentation techniques.

Conclusion

The RDA system demonstrates that "Human-AI Teaming" significantly enhances the speed and accuracy of disaster damage assessment. By combining the processing scale of AI with the nuanced judgment of human experts, the system successfully identified over 11,000 instances of severe damage during Hurricane Dorian. Future work will explore multi-modal approaches, integrating text and image data, and active learning techniques to further optimize the human-in-the-loop process.

Reference: Imran, M., Qazi, U., Ofli, F., Peterson, S., & Alam, F. (2022). AI for Disaster Rapid Damage Assessment from Microblogs. Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), 12517-12525.

Summary updated

BibTeX
@inproceedings{imran2022ai,
  author = {Muhammad Imran and Umair Qazi and Ferda Ofli and Steve Peterson and Firoj Alam},
  title = {{AI for Disaster Rapid Damage Assessment from Microblogs}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i11.21521},
}
Figure