Regression on Deep Visual Features using Artificial Neural Networks (ANNs) to Predict Hydraulic Blockage at Culverts

Wanqing Li, Umair Iqbal, Johan Barthélemy, Wanqing Li 0001, Pascal PerezarXiv 2021 · arXiv

Introduction

Urban infrastructure, particularly culverts, plays a vital role in flood mitigation. However, these structures are prone to blockage by debris such as vegetation and urban waste, which can lead to catastrophic flash flooding. Current management strategies typically rely on "visual blockage"—the percentage of the culvert opening obscured by debris.

A critical limitation of this approach is the disconnect between visual appearance and actual flow reduction, known as "hydraulic blockage." For instance, a culvert 80% covered by porous branches may still allow significant water passage, whereas a 20% blockage by solid plastic could severely impede flow. Historically, no quantifiable relationship has existed to translate visual data into hydraulic performance metrics. This research addresses this gap by proposing a deep learning-based pipeline to predict hydraulic blockage directly from images.

Methodology

The researchers developed an end-to-end machine learning pipeline designed to map high-dimensional visual features to a continuous hydraulic blockage percentage. The process involves three distinct stages:

  1. Deep Visual Feature Extraction: The system utilizes pre-trained Convolutional Neural Networks (CNNs)—specifically MobileNet, ResNet50, and EfficientNetB3—as feature extractors. By removing the final classification layers, these models transform raw culvert images into numerical feature vectors.
  2. Data Pre-processing: Extracted features undergo normalization via Standard Scalar transformation. This ensures that the data has a mean of zero and a standard deviation of one, which is essential for stable and efficient training of the subsequent regression model.
  3. ANN Regression: The normalized features are fed into an Artificial Neural Network (ANN). The study evaluated three ANN architectures (ANN1, ANN2, and ANN3) with depths ranging from 5 to 10 hidden layers to determine the optimal complexity for different CNN feature sets.
Overview of the proposed machine learning pipeline for predicting hydraulic blockage from culvert images.

Experimental Setup and Datasets

Due to the scarcity of high-quality visual and hydraulic data from real-world flood events, the study utilized a controlled laboratory environment to build a foundational dataset.

  • Hydrology-Lab Dataset (HD): This dataset was generated using a 12-meter flume equipped with scaled culvert models. It comprises 352 hydraulic data points across 173 unique blockage scenarios involving both vegetative and urban debris.
  • Visual Hydrology-Lab Dataset (VHD): Synchronized with the hydraulic measurements, this dataset contains 352 images extracted from video footage, providing the visual ground truth for each hydraulic state.
Schematic of the experimental flume setup used to collect the Hydrology-Lab and Visual Hydrology-Lab datasets.

The experimental setup included upstream and downstream point gauges to measure water levels, allowing for the precise calculation of hydraulic blockage based on flow capacity reduction relative to the visual state captured by the camera.

Results and Discussion

The study yielded several significant insights into the application of deep learning for hydrological monitoring:

Model Performance

The most effective configuration was the combination of MobileNet features with a 5-layer ANN (ANN1). This model achieved an $R^2$ score of 0.7855, demonstrating a strong correlation between visual features and hydraulic behavior. The Mean Absolute Error (MAE) was 4.67, indicating that the model's predictions were, on average, within approximately 5% of the actual measured hydraulic blockage.

Feature Complexity vs. Performance

An unexpected finding was that MobileNet, the simplest architecture tested, outperformed the deeper ResNet50 and EfficientNetB3 models. The authors hypothesize that the larger models extracted a high volume of redundant or uncorrelated features, introducing noise that hindered the regression task. This suggests that for specific infrastructure monitoring tasks, "less is more" regarding model depth.

Computational Efficiency

MobileNet also proved to be the most computationally efficient, with a total inference time of approximately 1.25 seconds per image. This high speed, combined with its lower memory footprint, makes the pipeline a strong candidate for deployment on "edge" devices—smart cameras capable of real-time processing at remote culvert sites.

Conclusion and Future Directions

This research provides empirical evidence that visual data contains sufficient information to predict hydraulic flow reduction, bridging a long-standing gap in flood management. However, several challenges remain for real-world implementation. The current models were trained in a controlled lab environment; real-world deployment will require models robust to varying lighting, weather conditions, and complex backgrounds. Future work should focus on collecting "in-the-wild" datasets from actual flood events and refining camera placement to minimize background interference. Despite these hurdles, this pipeline represents a significant step toward automated, data-driven flood risk assessment.

Summary updated

Abstract

Cross drainage hydraulic structures (i.e., culverts, bridges) in urban landscapes are prone to getting blocked by transported debris which often results in causing the flash floods. In context of Australia, Wollongong City Council (WCC) blockage conduit policy is the only formal guideline to consider blockage in design process. However, many argue that this policy is based on the post floods visual inspections and hence can not be considered accurate representation of hydraulic blockage. As a result of this on-going debate, visual blockage and hydraulic blockage are considered two distinct terms with no established quantifiable relation among both. This paper attempts to relate both terms by proposing the use of deep visual features for prediction of hydraulic blockage at a given culvert. An end-to-end machine learning pipeline is propounded which takes an image of culvert as input, extract visual features using deep learning models, pre-process the visual features and feed into regression model to predict the corresponding hydraulic blockage. Dataset (i.e., Hydrology-Lab Dataset (HD), Visual Hydrology-Lab Dataset (VHD)) used in this research was collected from in-lab experiments carried out using scaled physical models of culverts where multiple blockage scenarios were replicated at scale. Performance of regression models was assessed using standard evaluation metrics. Furthermore, performance of overall machine learning pipeline was assessed in terms of processing times for relative comparison of models and hardware requirement analysis. From the results ANN used with MobileNet extracted visual features achieved the best regression performance with $R^{2}$ score of 0.7855. Positive value of $R^{2}$ score indicated the presence of correlation between visual features and hydraulic blockage and suggested that both can be interrelated with each other.
BibTeX
@misc{li2021regression,
  author = {Wanqing Li and Umair Iqbal and Johan Barthélemy and Wanqing Li and Pascal Perez},
  title = {{Regression on Deep Visual Features using Artificial Neural Networks (ANNs) to Predict Hydraulic Blockage at Culverts}},
  year = {2021},
  eprint = {2105.03233},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2105.03233},
}
Figure