Abstract
Flood Insights is an end-to-end situational awareness system designed to bridge the gap in disaster response by fusing satellite imagery with social media sensing. By integrating deep learning-based computer vision for remote sensing and natural language processing for social media text and images, the system provides real-time, granular mapping of flood extent, infrastructure damage, and urgent humanitarian needs. During the 2022 Pakistan floods, the system demonstrated that combining social sensing with remote sensing significantly improves the correlation with ground-truth damage ($R^2$ increasing from 0.35 to 0.78), offering a holistic view of both physical and socio-economic impacts.
Introduction
During major flooding events, humanitarian organizations often face a critical "situational awareness gap." Traditional ground-truth surveys are slow, resource-heavy, and often impossible in inaccessible areas. While satellite remote sensing effectively maps water extent, it often lacks the human context necessary for prioritizing aid—such as identifying specific medical emergencies or localized infrastructure failures. This research addresses this by integrating "non-traditional" data from social media to complement satellite observations, creating a comprehensive disaster response tool that captures both physical destruction and immediate human needs.
Methods
The system architecture comprises three parallel processing pipelines designed for real-time data ingestion and analysis:
- Satellite & GIS Pipeline: Utilizes Sentinel-1 (SAR) and Sentinel-2 (Optical) imagery with a U-Net architecture for water segmentation. It incorporates nighttime light data for urban/rural classification and WorldPop datasets for demographic exposure analysis (age and gender).
- Social Media Text Pipeline: Processes tweets using a BERT-based model for toponym extraction and a BART-large transformer for zero-shot classification into a 38-class taxonomy of needs (e.g., shelter, water, medicine) and damages.
- Social Media Image Pipeline: Employs ResNet-50 for relevancy filtering to remove noise (memes/ads) and an Incidents model to classify specific damage types, such as collapsed buildings or ruptured roads, from user-uploaded photos.
Results and Key Insights
The system was deployed during the 2022 Pakistan floods, analyzing 1,608 satellite images and over 9.4 million tweets. Key findings include:
- Data Fusion Synergy: Remote sensing alone showed a weak correlation ($R^2 = 0.3483$) with official damage reports. Integrating social sensing data increased the explanatory power to $R^2 = 0.7824$, highlighting social media's role as a vital "human sensor" that fills gaps satellites cannot see.
- Impact Assessment: The system identified that 11.48% of Pakistan was flooded, exposing 18.95 million people. It surfaced over 20,000 shelter requests and 19,000 rescue/donation requests, providing a prioritized list of hard-hit districts to local authorities.
- Demographic Granularity: Exposure was broken down by demographics, revealing high vulnerability among children and the elderly. For example, in the Larkana district, these groups made up nearly half of the 1.26 million people exposed.
The integrated dashboard provides responders with a real-time, filterable interface to visualize these multi-source insights across different provinces and districts.
Conclusion
While the system significantly enhances situational awareness, it faces challenges such as the "digital divide" in rural areas where social media activity is lower. Future work will focus on optimizing recomputation latency for dynamic area-of-interest shifts and navigating evolving social media API constraints to ensure the long-term viability of real-time deployments.
Summary updated