Navigational Safety of Marine Surface Vehicles in Restricted Visibility: From Intelligent Vision to Vision-Centered Multi-Sensor Fusion

Ryan Wen Liu, Fengchuan Song, Tianxiang Wang, Guobei Peng, Youngrong Kim, Xian Zhong, Hooi-Siang Kang, Maohan Liang, Junzhi Yu, Shanliang YaoarXiv 2026 · arXiv

TL;DR

This paper provides a comprehensive survey of perception technologies for Marine Surface Vehicles (MSVs) operating in "restricted visibility"—conditions like dense fog, heavy rain, and low light. It charts the evolution from single-modality intelligent vision to sophisticated vision-centered multi-sensor fusion, offering a roadmap for achieving robust autonomous maritime navigation in adverse environments.

What Problem Does This Paper Address?

While autonomous driving on roads has seen explosive growth, the maritime domain presents a unique set of challenges. Marine Surface Vehicles (MSVs) rely heavily on shipboard cameras due to their high resolution and low cost. However, maritime environments are unforgiving: salt spray, irregular platform motion (pitch and roll), and adverse weather frequently degrade visual data.

Existing research often focuses on clear-weather scenarios or single-sensor solutions. This paper addresses the critical gap in navigational safety under restricted visibility. It explores how to simulate degraded maritime data for training, how to enhance poor-quality images, and how to fuse vision with complementary sensors like Radar, LiDAR, and the Automatic Identification System (AIS) to ensure MSVs don't just "see," but truly understand their surroundings.

Methods

The authors categorize the technical landscape into three main pillars:

  1. Image Generation in Restricted Visibility: To train robust AI, researchers need data. The paper discusses Physics-Based Methods (using models like Koschmieder’s law for haze or Retinex theory for low light) and Data-Driven Methods (using GANs and Diffusion Models) to synthesize realistic adverse-weather images.

    Figure 3: Overview of visually degraded image generation methods. Existing methods are mainly divided into two categories, i.e., physical modeling and data-driven methods.
  2. Intelligent Vision Modules: The paper reviews the "front-line" algorithms for processing visual data:

  3. Stabilization: Using horizon-line detection and gimbals to counteract sea-induced camera shake.
  4. Enhancement: Dehazing and low-light recovery techniques to restore visibility.
  5. Detection & Tracking: Adapting architectures like YOLO and DETR for the maritime context, focusing on scale-aware and lightweight models.
  6. Segmentation: Identifying "navigable water" vs. obstacles, often using IMU data to help the model understand the horizon.

    Figure 4: Overview of intelligent vision for marine surface vehicles. The intelligent vision pipeline integrates shipboard video stabilization, visual perception enhancement (including hazy, low-light, and multi-scene enhancement), object detection and tracking, and navigable area segmentation, providing robust situational awareness for MSVs in restricted visibility.
  7. Vision-Centered Multi-Sensor Fusion: This is the core of modern MSV safety. The authors examine how vision acts as the "anchor" for other sensors:

  8. Radar-Vision: Fusing the semantic richness of cameras with the precise range/velocity data of radar.
  9. LiDAR-Vision: Aligning 3D point clouds with 2D images for accurate spatial localization.
  10. AIS-Vision: Correlating visual detections with radio-frequency vessel identity reports to create an Augmented Reality (AR) navigation view.

Datasets

The paper tracks the evolution of maritime datasets through three stages:

  • Stage 1 (Early Benchmarks): Focused on monocular/stereo vision for basic tasks like horizon estimation (e.g., MODD, SMD).
  • Stage 2 (Large-scale RGB): Diverse, high-resolution datasets for panoptic scene understanding (e.g., LaRS, WUTDet).
  • Stage 3 (Multimodal & Visibility-aware): The current frontier, featuring synchronized data from 4D mmWave Radar, LiDAR, and Thermal cameras under adverse conditions (e.g., WaterScenes, USVTrack).

The authors maintain a curated list of these datasets at https://github.com/MAPGroup/Datasets.

Key Insights and Ideas

  • Modality Complementarity is King: Vision provides the "what" (semantics), Radar/LiDAR provide the "where" (geometry), and AIS provides the "who" (identity). Safety in fog requires all three.
  • The "All-in-One" Shift: There is a move away from specific-purpose enhancement (just dehazing) toward multi-scene models (like AiOENet) that can handle haze, rain, and low light simultaneously.
  • Physics-AI Hybridization: The most effective models for restricted visibility often incorporate physical imaging models (like atmospheric scattering) into deep learning architectures rather than relying on pure black-box data.
  • Edge Constraints: Unlike server-side AI, MSV perception must be lightweight. The paper highlights the trend toward depthwise separable convolutions and model pruning to fit AI onto small, power-constrained unmanned vessels.

Key Results

The survey identifies YOLO and DETR as the dominant frameworks for maritime object detection.

Representative YOLO- and DETR-based frameworks for maritime object detection

Qualitatively, the paper demonstrates that vision-centered fusion significantly reduces "false negatives" in restricted visibility. For instance, AIS-vision fusion allows a system to "see" a ship behind a fog bank by projecting its AIS coordinates onto the camera feed as an AR overlay, a technique already being commercialized by companies like Furuno and Orca AI.

Limitations and Future Work

Despite rapid progress, several "open challenges" remain:

  • Small Object Detection: Detecting a distant buoy or a small wooden boat in heavy swells remains extremely difficult for current sensors.
  • Uncertainty-Aware Fusion: Most current systems assume sensors are working perfectly. Future research needs to focus on "reliability allocation"—dynamically ignoring a blinded camera while trusting the Radar.
  • Scene Reasoning: Moving beyond simple detection to "interactive maritime cognition" using Multimodal Large Language Models (MLLMs) to interpret complex COLREGs (Collision Regulations) in real-time.
  • Lack of Extreme Weather Data: There is still a shortage of high-quality, synchronized multimodal datasets captured in truly severe maritime storms.

Summary updated

Abstract

Marine surface vehicles (MSVs) have attracted considerable attention for their commercial, scientific, and military applications. Owing to their favorable performance-to-cost ratio, shipboard cameras are widely used to detect and recognize potentially hazardous situations across diverse operational scenarios. However, visual perception is inevitably degraded by restricted conditions, including irregular motion of the imaging platform, adverse weather, and long imaging distances. To improve situational awareness and navigation capabilities, complementary sensors, such as the automatic identification system (AIS), radar, and light detection and ranging (LiDAR), etc., have become essential shipboard equipment. Multi-sensor fusion can provide richer perceptual information and help ensure navigational safety of MSVs in restricted visibility. This article reviews recent progress from intelligent vision to vision-centered multi-sensor fusion for newcomers and researchers in intelligent ship navigation and intelligent waterborne transportation system (IWTS). After introducing the generation of quality-degraded images in restricted visibility, we will review intelligent vision methods for four key issues, i.e., shipboard video stabilization, visual perception enhancement, object detection and tracking, and navigable area segmentation. We then examine vision-centered multi-sensor fusion methods, including infrared-visible image fusion, radar-vision fusion, LiDAR-vision fusion, AIS-vision fusion, and other multi-sensor fusion scenarios. Open-source datasets and safety-critical applications are also summarized and discussed. Finally, we highlight open challenges and future research directions. The collected maritime datasets are available at https://github.com/MAPGroup/Datasets and will be regularly updated.
BibTeX
@misc{liu2026navigational,
  author = {Ryan Wen Liu and Fengchuan Song and Tianxiang Wang and Guobei Peng and Youngrong Kim and Xian Zhong and Hooi-Siang Kang and Maohan Liang and Junzhi Yu and Shanliang Yao},
  title = {{Navigational Safety of Marine Surface Vehicles in Restricted Visibility: From Intelligent Vision to Vision-Centered Multi-Sensor Fusion}},
  year = {2026},
  eprint = {2611.05818},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.05818},
}
Figure