Real-Time Water Surface Object Detection Based on Improved Faster R-CNN

Lili Zhang, Yi Zhang, Zhen Zhang, Jie Shen, Huibin WangSensors 2019 · DOI

TL;DR

This paper presents an improved Faster R-CNN model for real-time detection of floating objects on water surfaces. By incorporating feature fusion and optimizing anchor settings based on dataset analysis, the method achieves a high mean average precision (MAP) of 83.7% with a detection speed of 13 FPS, making it suitable for practical environmental monitoring applications.

What Problem Does This Paper Address?

Detecting floating debris on water surfaces is crucial for environmental monitoring and maintaining water quality, especially with initiatives like China's "River Chief System." Traditional computer vision methods like background subtraction and image segmentation struggle with the dynamic nature of water surfaces (sunlight variations, waves) and lack generalization. While deep learning methods have shown promise, accurately detecting small, multi-scale objects in complex natural scenes in real-time remains a challenge. This paper aims to bridge this gap by proposing a robust and efficient deep learning approach for this specific task.

Methods

The core of the proposed method is an enhanced Faster R-CNN architecture. Key modifications include:

  • Gamma Correction for Preprocessing: To mitigate the impact of varying sunlight conditions that can lead to over- or under-exposed images, a gamma correction algorithm is applied. This enhances image contrast and makes it easier to distinguish floats from the background.
  • Feature Fusion: Instead of solely relying on high-level features, the model fuses low-level (detailed) and high-level (semantic) features. This is achieved by selecting appropriate convolutional layers (conv4_3 and conv5_3) based on their receptive field sizes, which are analyzed to match object scales. This fusion helps in detecting small objects more effectively.
  • Optimized Anchor Settings: The authors analyze the distribution of object scales and aspect ratios in their custom dataset. Based on this analysis, they propose custom anchor scales, aspect ratios, and numbers for the Region Proposal Network (RPN) to better cover the diverse sizes of water surface floats, improving robustness and accuracy for multi-scale objects.
  • Alternating Optimization Training: The network is trained using an alternating optimization strategy, involving sequential training and fine-tuning of the RPN and the Fast R-CNN components.

Datasets

The authors constructed their own dataset for training and evaluation.

  • Source: Video surveillance streams from the North Canal in Beijing over three days.
  • Initial Data: 1065 images obtained by frame interception.
  • Data Enhancement: The dataset was expanded using rotation and random clipping techniques, resulting in a total of 2420 images.
  • Splitting: The dataset was divided into training and testing sets with a 7:3 ratio.
  • Annotation: Images were annotated using the LabelImg tool in the VOC2007 format, with bounding boxes and category names for floats.

Key Insights and Ideas

  • Sunlight is a Major Hurdle: The paper highlights how variable lighting conditions significantly degrade the performance of object detection on water surfaces, underscoring the need for effective preprocessing.
  • Feature Fusion is Key for Small Objects: Combining low-level and high-level features is crucial for detecting small objects, as low-level features provide necessary detail while high-level features offer semantic context. The choice of which layers to fuse matters and can be guided by receptive field analysis.
  • Dataset-Specific Anchors Matter: Generic anchor settings in object detection frameworks like Faster R-CNN are not optimal for all tasks. Analyzing the specific object scale distribution in a dataset allows for tailored anchor configurations that significantly boost performance.
  • Real-time Performance is Achievable: While feature fusion adds computational cost, the proposed method demonstrates that a balance between high accuracy and near real-time detection (11-13 FPS) is attainable for this application.

Key Results

  • Accuracy: The proposed method achieved a Mean Average Precision (MAP) of 83.7% on their custom dataset.
  • Speed: The detection speed was 11-13 frames per second (FPS).
  • Comparison:
    • Outperformed standard Faster R-CNN by 2.5% MAP (81.2% vs. 83.7%) when using feature fusion.
  • Significantly improved upon Fast R-CNN (by ~8%) and YOLOv3 (from 78.6% to 83.7% MAP).
  • While YOLOv3 was faster (35 FPS), the proposed method's speed was deemed sufficient for the application.
  • Qualitative: Visualizations showed the proposed method successfully detecting both large and small floats, even in challenging conditions, whereas YOLOv3 missed some small objects.

Limitations and Future Work

  • Dataset Limitations: The current dataset contains only one category of floating objects, limiting generalization for broader engineering applications.
  • Future Work: Collect diverse datasets including various types of domestic garbage.
  • Image Enhancement: The gamma correction method requires manual determination of the gamma value based on the dataset.
  • Future Work: Explore more effective and generalizable image enhancement techniques.
  • Detection Speed: Feature fusion, while beneficial for accuracy, increases computational load and reduces speed.
  • Future Work: Optimize the model further to improve detection speed.
  • Large-Scale Floats: For particularly large floats, semantic segmentation might be more appropriate.
  • Future Work: Consider using Mask R-CNN for semantic segmentation to quantify float areas.

Summary updated

Abstract

In this paper, we consider water surface object detection in natural scenes. Generally, background subtraction and image segmentation are the classical object detection methods. The former is highly susceptible to variable scenes, so its accuracy will be greatly reduced when detecting water surface objects due to the changing of the sunlight and waves. The latter is more sensitive to the selection of object features, which will lead to poor generalization as a result, so it cannot be applied widely. Consequently, methods based on deep learning have recently been proposed. The River Chief System has been implemented in China recently, and one of the important requirements is to detect and deal with the water surface floats in a timely fashion. In response to this case, we propose a real-time water surface object detection method in this paper which is based on the Faster R-CNN. The proposed network model includes two modules and integrates low-level features with high-level features to improve detection accuracy. Moreover, we propose to set the different scales and aspect ratios of anchors by analyzing the distribution of object scales in our dataset, so our method has good robustness and high detection accuracy for multi-scale objects in complex natural scenes. We utilized the proposed method to detect the floats on the water surface via a three-day video surveillance stream of the North Canal in Beijing, and validated its performance. The experiments show that the mean average precision (MAP) of the proposed method was 83.7%, and the detection speed was 13 frames per second. Therefore, our method can be applied in complex natural scenes and mostly meets the requirements of accuracy and speed of water surface object detection online.
BibTeX
@inproceedings{zhang2019real,
  author = {Lili Zhang and Yi Zhang and Zhen Zhang and Jie Shen and Huibin Wang},
  title = {{Real-Time Water Surface Object Detection Based on Improved Faster R-CNN}},
  booktitle = {Sensors},
  year = {2019},
  doi = {10.3390/s19163523},
}
Figure