TL;DR
This paper provides a comprehensive survey of perception technologies for Marine Surface Vehicles (MSVs) operating in "restricted visibility"—conditions like dense fog, heavy rain, and low light. It charts the evolution from single-modality intelligent vision to sophisticated vision-centered multi-sensor fusion, offering a roadmap for achieving robust autonomous maritime navigation in adverse environments.
What Problem Does This Paper Address?
While autonomous driving on roads has seen explosive growth, the maritime domain presents a unique set of challenges. Marine Surface Vehicles (MSVs) rely heavily on shipboard cameras due to their high resolution and low cost. However, maritime environments are unforgiving: salt spray, irregular platform motion (pitch and roll), and adverse weather frequently degrade visual data.
Existing research often focuses on clear-weather scenarios or single-sensor solutions. This paper addresses the critical gap in navigational safety under restricted visibility. It explores how to simulate degraded maritime data for training, how to enhance poor-quality images, and how to fuse vision with complementary sensors like Radar, LiDAR, and the Automatic Identification System (AIS) to ensure MSVs don't just "see," but truly understand their surroundings.
Methods
The authors categorize the technical landscape into three main pillars:
-
Image Generation in Restricted Visibility: To train robust AI, researchers need data. The paper discusses Physics-Based Methods (using models like Koschmieder’s law for haze or Retinex theory for low light) and Data-Driven Methods (using GANs and Diffusion Models) to synthesize realistic adverse-weather images.
Figure 3: Overview of visually degraded image generation methods. Existing methods are mainly divided into two categories, i.e., physical modeling and data-driven methods. -
Intelligent Vision Modules: The paper reviews the "front-line" algorithms for processing visual data:
- Stabilization: Using horizon-line detection and gimbals to counteract sea-induced camera shake.
- Enhancement: Dehazing and low-light recovery techniques to restore visibility.
- Detection & Tracking: Adapting architectures like YOLO and DETR for the maritime context, focusing on scale-aware and lightweight models.
-
Segmentation: Identifying "navigable water" vs. obstacles, often using IMU data to help the model understand the horizon.
Figure 4: Overview of intelligent vision for marine surface vehicles. The intelligent vision pipeline integrates shipboard video stabilization, visual perception enhancement (including hazy, low-light, and multi-scene enhancement), object detection and tracking, and navigable area segmentation, providing robust situational awareness for MSVs in restricted visibility. -
Vision-Centered Multi-Sensor Fusion: This is the core of modern MSV safety. The authors examine how vision acts as the "anchor" for other sensors:
- Radar-Vision: Fusing the semantic richness of cameras with the precise range/velocity data of radar.
- LiDAR-Vision: Aligning 3D point clouds with 2D images for accurate spatial localization.
- AIS-Vision: Correlating visual detections with radio-frequency vessel identity reports to create an Augmented Reality (AR) navigation view.
Datasets
The paper tracks the evolution of maritime datasets through three stages:
- Stage 1 (Early Benchmarks): Focused on monocular/stereo vision for basic tasks like horizon estimation (e.g., MODD, SMD).
- Stage 2 (Large-scale RGB): Diverse, high-resolution datasets for panoptic scene understanding (e.g., LaRS, WUTDet).
- Stage 3 (Multimodal & Visibility-aware): The current frontier, featuring synchronized data from 4D mmWave Radar, LiDAR, and Thermal cameras under adverse conditions (e.g., WaterScenes, USVTrack).
The authors maintain a curated list of these datasets at https://github.com/MAPGroup/Datasets.
Key Insights and Ideas
- Modality Complementarity is King: Vision provides the "what" (semantics), Radar/LiDAR provide the "where" (geometry), and AIS provides the "who" (identity). Safety in fog requires all three.
- The "All-in-One" Shift: There is a move away from specific-purpose enhancement (just dehazing) toward multi-scene models (like AiOENet) that can handle haze, rain, and low light simultaneously.
- Physics-AI Hybridization: The most effective models for restricted visibility often incorporate physical imaging models (like atmospheric scattering) into deep learning architectures rather than relying on pure black-box data.
- Edge Constraints: Unlike server-side AI, MSV perception must be lightweight. The paper highlights the trend toward depthwise separable convolutions and model pruning to fit AI onto small, power-constrained unmanned vessels.
Key Results
The survey identifies YOLO and DETR as the dominant frameworks for maritime object detection.
Qualitatively, the paper demonstrates that vision-centered fusion significantly reduces "false negatives" in restricted visibility. For instance, AIS-vision fusion allows a system to "see" a ship behind a fog bank by projecting its AIS coordinates onto the camera feed as an AR overlay, a technique already being commercialized by companies like Furuno and Orca AI.
Limitations and Future Work
Despite rapid progress, several "open challenges" remain:
- Small Object Detection: Detecting a distant buoy or a small wooden boat in heavy swells remains extremely difficult for current sensors.
- Uncertainty-Aware Fusion: Most current systems assume sensors are working perfectly. Future research needs to focus on "reliability allocation"—dynamically ignoring a blinded camera while trusting the Radar.
- Scene Reasoning: Moving beyond simple detection to "interactive maritime cognition" using Multimodal Large Language Models (MLLMs) to interpret complex COLREGs (Collision Regulations) in real-time.
- Lack of Extreme Weather Data: There is still a shortage of high-quality, synchronized multimodal datasets captured in truly severe maritime storms.
Summary updated