Floodborne Objects Type Recognition Using Computer Vision to Mitigate Blockage Originated Floods

Umair Iqbal, Muhammad Zain Bin Riaz, Johan Barthélemy, Nathanael Hutchison, Pascal PerezWater 2022 · DOI

TL;DR

This paper introduces an automated system for identifying floodborne objects (vegetation and urban debris) using computer vision. By training deep learning models on a new dataset, they achieve high accuracy in distinguishing object types, which is crucial for improving flood prediction and mitigation strategies that currently lack this capability.

What Problem Does This Paper Address?

Floods are devastating natural disasters, and a significant contributing factor to their severity, especially in urban areas, is the blockage of hydraulic structures like culverts and bridges by debris. This debris, ranging from vegetation to urban waste, significantly alters water flow, leading to increased velocities, scouring, and structural damage. Current flood models struggle to account for these complex, non-linear interactions because identifying and categorizing floodborne objects is a manual, time-consuming process. This paper aims to bridge this gap by developing an automated method for recognizing floodborne object types, which is a critical first step towards integrating this information into more accurate flood models and management systems.

Methods

The core of this research lies in applying state-of-the-art computer vision object detection models. Specifically, the authors trained and evaluated two popular deep learning architectures: Faster R-CNN and YOLOv4.

  • Faster R-CNN is a two-stage detector that first proposes regions of interest (RPNs) and then classifies objects within those regions. The paper uses variants with different backbones (MobileNet and ResNet18) to explore performance trade-offs.
  • YOLOv4 is a single-stage detector known for its speed and accuracy, also tested with different backbones (ResNet18 and CSPDarkNet).

The training process involved standard deep learning practices such as data augmentation, optimization functions (SGD, Adam), and regularization techniques. Performance was evaluated using standard metrics like Mean Average Precision (mAP) and Average Precision (AP) for each class.

Datasets

The study introduces a novel dataset called the Floodborne Objects Recognition Dataset (FORD). This dataset is a combination of:

  • Real images: Sourced from historical records of the Wollongong City Council (WCC), showing actual floodborne objects blocking hydraulic structures.
  • Simulated images: Generated from laboratory experiments using scaled physical models of culverts with various blockage scenarios.

The FORD dataset contains 663 images (141 real, 522 simulated) with 946 bounding box annotations, categorized into "vegetated" (640) and "urban" (306) objects. This dataset is crucial as it addresses the lack of existing benchmarks for floodborne object type recognition.

Key Insights and Ideas

  • The critical need for object type: The paper strongly emphasizes that different types of floodborne objects (vegetation vs. urban debris) have distinct hydrodynamic impacts. Recognizing these types is not just an academic exercise but a practical necessity for effective flood management.
  • Deep learning's potential: The research demonstrates that deep learning models, particularly Faster R-CNN, can effectively learn to distinguish between vegetation and urban objects in complex, real-world scenarios, overcoming the limitations of manual identification.
  • Dataset composition matters: The combination of real and simulated data in FORD was essential. While real data provides authenticity, simulated data allowed for the creation of a sufficiently large and diverse dataset to train data-hungry deep learning models.
  • Urban objects are easier to detect: The results consistently show higher detection accuracy (AP) for urban objects compared to vegetation. This is attributed to the more distinct visual characteristics of urban debris and potentially less background noise interference compared to vegetation, which can blend with natural surroundings.

Key Results

The study found that the Faster R-CNN model with a MobileNet backbone performed best, achieving a Mean Average Precision (mAP) of 84% on the test dataset.

  • Class-specific performance: The best model achieved an AP of 75% for vegetated objects and a significantly higher AP of 93% for urban objects.
  • Model comparison: Faster R-CNN models generally outperformed YOLOv4 models in terms of mAP.
  • Training time: The Faster R-CNN with MobileNet was also the fastest to train, taking only 30 seconds per epoch.
  • Qualitative results: The paper includes examples of both correct and incorrect predictions, highlighting that while the model is effective, challenges remain, particularly with distinguishing vegetation from similar natural backgrounds.

Limitations and Future Work

  • Data availability: The authors acknowledge that the availability of relevant visual data from real flooding events is a significant challenge.
  • Vegetation detection: The lower accuracy in detecting vegetation is a noted limitation, likely due to its visual similarity to natural backgrounds and cluttered scenes.
  • Future directions:
    • Synthetic data generation: Exploring advanced techniques like GANs and NVIDIA Isaac Replicator to create more diverse and realistic synthetic datasets.
  • Hybrid models: Integrating computer vision outputs with existing flood models for more adaptive and accurate flood forecasting.
  • Sub-class detection: Moving beyond broad categories to identify specific types of vegetation (e.g., branches, logs) and urban objects (e.g., cars, bins).
  • Quantification: Developing methods to estimate the volume or mass of detected floodborne objects, aligning with existing literature that focuses on debris budgeting.
  • Pilot deployments: Implementing and fine-tuning these systems in real-world scenarios to gather more data and improve performance.

Summary updated

Abstract

The presence of floodborne objects (i.e., vegetation, urban objects) during floods is considered a very critical factor because of their non-linear complex hydrodynamics and impacts on flooding outcomes (e.g., diversion of flows, damage to structures, downstream scouring, failure of structures). Conventional flood models are unable to incorporate the impact of floodborne objects mainly because of the highly complex hydrodynamics and non-linear nature associated with their kinematics and accumulation. Vegetation (i.e., logs, branches, shrubs, entangled grass) and urban objects (i.e., vehicles, bins, shopping carts, building waste materials) offer significant materialistic, hydrodynamic and characterization differences which impact flooding outcomes differently. Therefore, recognition of the types of floodborne objects is considered a key aspect in the process of assessing their impact on flooding. The identification of floodborne object types is performed manually by the flood management officials, and there exists no automated solution in this regard. This paper proposes the use of computer vision technologies for automated floodborne objects type identification from a vision sensor. The proposed approach is to use computer vision object detection (i.e., Faster R-CNN, YOLOv4) models to detect a floodborne object’s type from a given image. The dataset used for this research is referred to as the “Floodborne Objects Recognition Dataset (FORD)” and includes real images of floodborne objects blocking the hydraulic structures extracted from Wollongong City Council (WCC) records and simulated images of scaled floodborne objects blocking the culverts collected from hydraulics laboratory experiments. From the results, the Faster R-CNN model with MobileNet backbone was able to achieve the best Mean Average Precision (mAP) of 84% over the test dataset. To demonstrate the practical use of the proposed approach, two potential use cases for the proposed floodborne object type recognition are reported. Overall, the performance of the implemented computer vision models indicated that such models have the potential to be used for automated identification of floodborne object types.
BibTeX
@inproceedings{iqbal2022floodborne,
  author = {Umair Iqbal and Muhammad Zain Bin Riaz and Johan Barthélemy and Nathanael Hutchison and Pascal Perez},
  title = {{Floodborne Objects Type Recognition Using Computer Vision to Mitigate Blockage Originated Floods}},
  booktitle = {Water},
  year = {2022},
  doi = {10.3390/w14172605},
}
Figure