Exploiting ConvNet Diversity for Flooding Identification

Keiller Nogueira, Samuel G. Fadel, Ícaro C. Dourado, Rafael de Oliveira Werneck, Javier A. V. Muñoz, Otávio A. B. Penatti, Rodrigo Tripodi Calumby, Lin Tzy Li, Jefersson A. dos Santos, Ricardo da Silva Torres

TL;DR

This paper tackles the critical problem of automatically identifying flooded areas in high-resolution satellite imagery using deep learning. The authors propose novel Convolutional Neural Network (ConvNet) architectures and a combination strategy that leverages the diversity of these networks to achieve state-of-the-art performance in flood detection, outperforming existing methods.

What Problem Does This Paper Address?

Flooding is a devastating natural disaster with significant economic and human costs. Effective flood monitoring is crucial for disaster preparedness, response, and victim assistance. A key step in this monitoring process is accurately identifying areas vulnerable to or currently experiencing flooding. While remote sensing offers a powerful way to capture large-scale inundation, automatically and accurately delineating these flooded regions from high-resolution imagery remains a challenge. Existing methods often struggle with the complex visual characteristics of flooded areas, and there's a need for robust deep learning approaches that can effectively learn from and generalize across diverse flooding events.

Methods

The core of this work lies in developing and combining specialized Convolutional Neural Networks (ConvNets) for flood identification. The authors introduce two main types of architectures:

  • Dilated Convolutions: These networks utilize dilated convolutional layers, which expand the receptive field of filters without downsampling the input image. This preserves spatial resolution, allowing the network to capture finer details crucial for identifying smaller flooded features.
  • Deconvolutional Networks: Inspired by architectures like SegNet, these networks employ an encoder-decoder structure. The encoder learns feature representations, and the decoder then upsamples these features to generate a dense prediction map at the original image resolution.

To further enhance performance, the authors propose a combination strategy that exploits the diversity of these individual networks. The premise is that different network architectures, trained on the same data, will learn complementary features. They concatenate the output prediction maps from multiple ConvNets and use a Support Vector Machine (SVM) to learn how to best combine these predictions, aiming to produce a more accurate final flood map. They also explored a majority voting (Fusion-MV) approach, but found the SVM-based fusion to be more effective.

Datasets

The primary dataset used in this research is a collection of satellite image patches from the 2017 Multimedia Satellite Task's Flood-Detection in Satellite Images subtask. This dataset consists of high-resolution image patches (320x320 pixels) with four bands (red, green, blue, and near infrared) and a ground-sample distance (GSD) of 3.7 meters. Each pixel is classified as either "background" or "flooded area."

The dataset was divided into:

  • Training set: 462 image patches from six different flooding events.
  • Internal validation set: 92 patches from the training set for hyperparameter tuning.
  • Test sets:
    • Same Locations test set: 216 unseen patches from the same regions as the training data.
  • New Locations test set: 58 unseen patches from regions not present in the training data.

The authors note that ground-truth information for the test sets was not released by the competition organizers until after the submission of their paper.

Key Insights and Ideas

  • Diversity is Key: Combining different ConvNet architectures (specifically dilated and deconvolutional ones) leads to significant performance improvements over individual networks. This highlights that no single architecture is optimal, and their complementarity is crucial.
  • Dilated Convolutions Preserve Detail: The use of dilated convolutions is effective for flood detection because it allows networks to capture fine-grained spatial information without losing resolution, which is vital for identifying the often irregular shapes of flooded areas.
  • SVM Fusion Outperforms Majority Voting: While combining predictions is beneficial, a learned approach like SVM is more robust than simple majority voting for aggregating diverse network outputs, suggesting that the SVM can better learn the nuances of when and how to trust each network's prediction.
  • Patch Size Matters (and Smaller is Better Here): Counterintuitively, using larger input patches (50x50) did not improve results and even decreased performance compared to smaller patches (25x25). The authors hypothesize this is due to the reduced amount of training data available for larger patches, emphasizing deep learning's need for ample labeled data.
  • Potential for General Water Segmentation: The visual results suggest that the proposed methods might also be suitable for general water segmentation, hinting at future work to distinguish between floodwater and normal surface water.

Key Results

The proposed methods achieved state-of-the-art performance on the Flood-Detection in Satellite Images subtask of the 2017 Multimedia Satellite Task, winning the challenge.

  • The best performing method was the Fusion-SVM strategy, which combined the predictions of all trained ConvNets.
  • This Fusion-SVM approach achieved improvements of 1-4% in Jaccard Index over the baselines on both test sets.
  • Specifically, on the Same Locations test set, the Fusion-SVM achieved an IOU of 88% (with 25x25 patches) and 84% (with 50x50 patches).
  • On the New Locations test set, the Fusion-SVM achieved an IOU of 84%.
  • The paper also demonstrated that training specialized ConvNets for each location (Location ConvNets) was beneficial for the "New Locations" test set, indicating that specific event characteristics can be learned.

Limitations and Future Work

  • Distinguishing Flood vs. Normal Water: The authors acknowledge that their current classification is binary (background vs. flooded area) and does not distinguish between floodwater and normal surface water (like rivers). They suggest that with appropriate training data, their methods could be extended to perform this distinction.
  • Dataset Size for Larger Patches: The decrease in performance with larger patch sizes suggests a need for more labeled data if larger receptive fields are to be effectively utilized.
  • Generalizability to More Datasets: While the results are strong on the provided dataset, the authors mention the intention to analyze the proposed method using more datasets to further evaluate its robustness and generalizability.
  • Exploring Other Combination Strategies: While SVM fusion was successful, further exploration of other machine learning techniques for combining network outputs could be beneficial.

Summary updated

Abstract

Flooding is the world's most costly type of natural disaster in terms of both economic losses and human causalities. A first and essential procedure toward flood monitoring is based on identifying the area most vulnerable to flooding, which gives authorities relevant regions to focus. In this letter, we propose several methods to perform flooding identification in high-resolution remote sensing images using deep learning. Specifically, some proposed techniques are based upon unique networks, such as dilated and deconvolutional ones, whereas others were conceived to exploit diversity of distinct networks in order to extract the maximum performance of each classifier. The evaluation of the proposed methods was conducted in a high-resolution remote sensing data set. Results show that the proposed algorithms outperformed the state-of-the-art baselines, providing improvements ranging from 1% to 4% in terms of the Jaccard Index.
BibTeX
@article{nogueira2018exploiting,
  author = {Keiller Nogueira and Samuel G. Fadel and Ícaro C. Dourado and Rafael de Oliveira Werneck and Javier A. V. Muñoz and Otávio A. B. Penatti and Rodrigo Tripodi Calumby and Lin Tzy Li and Jefersson A. dos Santos and Ricardo da Silva Torres},
  title = {{Exploiting ConvNet Diversity for Flooding Identification}},
  journal = {IEEE Geosci. Remote. Sens. Lett.},
  year = {2018},
  url = {https://openreview.net/forum?id=Dz0FuKlosSP},
}
Figure