Effective Waterline Detection of Unmanned Surface Vehicles Based on Optical Images

Yangjie Wei, Yuwei ZhangSensors 2016 · DOI

TL;DR

This paper introduces a novel, robust method for accurately detecting the waterline on optical images captured by Unmanned Surface Vehicles (USVs). By combining structure extraction to remove noise and texture analysis using Local Binary Patterns (LBPs) and Gray Level Co-occurrence Matrix (GLCM), the algorithm effectively handles challenging real-world conditions like varying illumination and motion, achieving an average error of just 1.84 pixels.

What Problem Does This Paper Address?

Autonomous navigation for Unmanned Surface Vehicles (USVs) in complex environments like rivers and harbors is a significant challenge. A critical component for safe and effective USV operation is the ability to accurately detect the boundary between water and land (the waterline). Existing methods for waterline detection using optical images often struggle with robustness and precision, especially in dynamic outdoor settings where factors like illumination changes, waves, and the USV's own motion introduce significant noise and visual complexity. This paper aims to bridge this gap by proposing a more effective and reliable waterline detection algorithm tailored for USV applications.

Methods

The proposed method consists of three main stages:

  1. Image Pre-processing (Structure Extraction): This initial step focuses on removing undesirable textures caused by environmental factors like illumination variations, wind, and USV motion. The authors employ a structure extraction technique (based on local variation measures) to preserve the essential structural features of the image while smoothing out noise. Following this, a simple background (sky) removal is performed by analyzing the histogram of the blue channel.
  2. First Segmentation (Texture Analysis): After preprocessing, the algorithm utilizes texture analysis to differentiate between water and land.
  3. Local Binary Patterns (LBPs): LBPs are computed to capture local texture information. This results in an 8-bit grayscale image representing texture patterns.
  4. Gray Level Co-occurrence Matrix (GLCM): To analyze the texture patterns from LBPs, GLCM is employed. It captures the spatial relationships of pixel intensities (in this case, LBP values) at different orientations and distances.
  5. Entropy Calculation: The entropy of the GLCM is used as a texture descriptor. By calculating the difference in entropy between adjacent image blocks, the algorithm identifies potential separation points along columns, aiming to distinguish water from land.
  6. Final Segmentation (Refinement): To improve the precision of the waterline detected in the first segmentation, a final refinement step is introduced.
  7. Hough Transformation: A straight line is detected from the LBP image using Hough transformation. This line serves as a reference.
  8. Local Searching Area: A local searching area is defined around the reference line.
  9. Compensation: If the first segmentation produces unusual peaks or inaccuracies, the algorithm searches within the defined local area to find a more accurate and smoother waterline, compensating for the limitations of solely relying on vertical texture comparisons.

Datasets

The paper does not explicitly mention the use of pre-existing public datasets. Instead, it relies on images captured by a USV system during experiments conducted in real-world inland river environments. These images were collected under various conditions, including different riverbank distances, sunlight levels, and shades, forming the basis for their evaluation.

Key Insights and Ideas

  • Structure Extraction is Crucial: Removing noise textures from motion, wind, and illumination before texture analysis is key to achieving robust waterline detection in dynamic environments.
  • Combining LBP and GLCM for Texture: Leveraging both LBP for local texture patterns and GLCM for their spatial relationships provides a more comprehensive texture description than either method alone.
  • Entropy as a Discriminator: The entropy of GLCM effectively captures the complexity and disorder of textures, allowing for differentiation between water and land.
  • Two-Stage Segmentation for Precision: A two-stage approach, starting with a rough texture-based segmentation and followed by a refinement step using a reference line and local search, significantly improves the accuracy and smoothness of the detected waterline.
  • Robustness to Environmental Factors: The proposed method demonstrates resilience to challenging conditions like uneven illumination and shadows, which often degrade the performance of simpler methods.

Key Results

The proposed method achieved promising results in detecting waterlines under various experimental conditions:

  • Average Error: The average error of the detected waterline was 1.84 pixels.
  • Mean Square Deviation: The mean square deviation was 4.57 pixels.
  • Qualitative Performance: The algorithm successfully detected waterlines in images with varying riverbank distances, sunlight conditions, and even in the presence of shade from trees. It was able to distinguish water from land even when the colors were similar or when mountains had intensities close to the water.
  • Comparison: When compared to edge detection and super pixel segmentation methods, the proposed algorithm showed superior performance, particularly in challenging scenarios with shade and uneven illumination. Edge detection struggled with shaded areas, and super pixel segmentation was highly sensitive to illumination.

Limitations and Future Work

  • Illumination Sensitivity: While the method is more robust than others, the authors acknowledge that under extremely bad illumination conditions (very dark environments where trees and shade are indistinguishable), the waterline detection can still have errors.
  • Future Work: The authors state their intention to further research and improve the performance of their waterline detection method, particularly to address the limitations encountered in very poor illumination scenarios.

Summary updated

Abstract

Real-time and accurate detection of the sailing or water area will help realize unmanned surface vehicle (USV) systems. Although there are some methods for using optical images in USV-oriented environmental modeling, both the robustness and precision of these published waterline detection methods are comparatively low for a real USV system moving in a complicated environment. This paper proposes an efficient waterline detection method based on structure extraction and texture analysis with respect to optical images and presents a practical application to a USV system for validation. First, the basic principles of local binary patterns (LBPs) and gray level co-occurrence matrix (GLCM) were analyzed, and their advantages were integrated to calculate the texture information of river images. Then, structure extraction was introduced to preprocess the original river images so that the textures resulting from USV motion, wind, and illumination are removed. In the practical application, the waterlines of many images captured by the USV system moving along an inland river were detected with the proposed method, and the results were compared with those of edge detection and super pixel segmentation. The experimental results showed that the proposed algorithm is effective and robust. The average error of the proposed method was 1.84 pixels, and the mean square deviation was 4.57 pixels.
BibTeX
@inproceedings{wei2016effective,
  author = {Yangjie Wei and Yuwei Zhang},
  title = {{Effective Waterline Detection of Unmanned Surface Vehicles Based on Optical Images}},
  booktitle = {Sensors},
  year = {2016},
  doi = {10.3390/s16101590},
}
Figure