Fast image-based obstacle detection from unmanned surface vehicles

Matej Kristan, Vildana Sulic, Stanislav Kovacic, Janez Pers

Abstract

Obstacle detection plays an important role in unmanned surface vehicles (USV). The USVs operate in highly diverse environments in which an obstacle may be a floating piece of wood, a scuba diver, a pier, or a part of a shoreline, which presents a significant challenge to continuous detection from images taken onboard. This paper addresses the problem of online detection by constrained unsupervised segmentation. To this end, a new graphical model is proposed that affords a fast and continuous obstacle image-map estimation from a single video stream captured onboard a USV. The model accounts for the semantic structure of marine environment as observed from USV by imposing weak structural constraints. A Markov random field framework is adopted and a highly efficient algorithm for simultaneous optimization of model parameters and segmentation mask estimation is derived. Our approach does not require computationally intensive extraction of texture features and comfortably runs in real-time. The algorithm is tested on a new, challenging, dataset for segmentation and obstacle detection in marine environments, which is the largest annotated dataset of its kind. Results on this dataset show that our model outperforms the related approaches, while requiring a fraction of computational effort.

Introduction

Navigating small Unmanned Surface Vehicles (USVs) in coastal waters presents unique computer vision challenges. Unlike ground vehicles with stable ground planes, USVs face shifting horizons, dynamic obstacles like buoys or swimmers, and extreme lighting conditions such as sun glitter. Furthermore, strict weight and power constraints often preclude the use of heavy GPUs or active sensors like LIDAR. This paper introduces a highly efficient, real-time obstacle detection approach that leverages a novel graphical model with weak semantic priors to achieve state-of-the-art performance on a single CPU core.

Methodology

The proposed system models the marine environment as a mixture of four components: three Gaussian Mixture Models (GMMs) representing the sky, land/haze, and water, plus a uniform distribution for outliers (obstacles).

Semantic Generative Model

The approach avoids computationally expensive texture features, instead utilizing simple color and pixel coordinates. It imposes "weak structural priors" based on the typical vertical arrangement of the marine environment: sky at the top, land in the middle, and water at the bottom.

The proposed pipeline: from input image to obstacle map.

Optimization and Speed

A Markov Random Field (MRF) ensures spatial smoothness in segmentation. The authors derive a specialized Expectation-Maximization (EM) algorithm where neighborhood interactions are computed via simple convolutions. This allows the system to run at over 70fps on a single core. To further enhance speed, main segmentation occurs on a reduced 50x50 pixel image before rescaling, maintaining detection of critical obstacles.

Experimental Results

The algorithm was evaluated on the Marine Obstacle Detection Dataset (MODD), featuring 12 video sequences with 4,454 annotated frames. This dataset includes challenging "extreme" conditions with intense sun glitter.

Performance Metrics

In "normal" conditions, the proposed Semantic Segmentation Model (SSM) achieved an F-score of 0.819, outperforming Graph-Cut Models (GCM) and standard MRF approaches. The system demonstrated remarkable robustness against sun glitter, maintaining a low false-positive rate where other methods failed.

Qualitative comparison showing the proposed method (SSM) producing much cleaner segmentation than competing approaches.

Conclusion

This work demonstrates that the inherent semantic structure of marine environments (sky-land-water) can be more effective than complex texture features for obstacle detection. By modeling obstacles as outliers to these dominant regions and utilizing efficient convolutional MRF optimization, real-time performance is achieved without specialized hardware. Future work includes integrating IMU data for better horizon estimation and GPU porting for even higher throughput.

Summary updated

Abstract

Obstacle detection plays an important role in unmanned surface vehicles (USV). The USVs operate in highly diverse environments in which an obstacle may be a floating piece of wood, a scuba diver, a pier, or a part of a shoreline, which presents a significant challenge to continuous detection from images taken onboard. This paper addresses the problem of online detection by constrained unsupervised segmentation. To this end, a new graphical model is proposed that affords a fast and continuous obstacle image-map estimation from a single video stream captured onboard a USV. The model accounts for the semantic structure of marine environment as observed from USV by imposing weak structural constraints. A Markov random field framework is adopted and a highly efficient algorithm for simultaneous optimization of model parameters and segmentation mask estimation is derived. Our approach does not require computationally intensive extraction of texture features and comfortably runs in real-time. The algorithm is tested on a new, challenging, dataset for segmentation and obstacle detection in marine environments, which is the largest annotated dataset of its kind. Results on this dataset show that our model outperforms the related approaches, while requiring a fraction of computational effort.
BibTeX
@article{kristan2016fast,
  author = {Matej Kristan and Vildana Sulic and Stanislav Kovacic and Janez Pers},
  title = {{Fast image-based obstacle detection from unmanned surface vehicles}},
  journal = {IEEE Trans. Cybernetics},
  year = {2016},
  doi = {10.1109/tcyb.2015.2412251},
}
Figure