Real-Time Multi-Scene Visibility Enhancement for Promoting Navigational Safety of Vessels Under Complex Weather Conditions

Ryan Wen Liu, Yuxu Lu, Yuan Gao, Yu Guo, Wenqi Ren, Fenghua Zhu, Fei-Yue Wang

Abstract

The visible-light camera, which is capable of environment perception and navigation assistance, has emerged as an essential imaging sensor for marine surface vessels in intelligent waterborne transportation systems (IWTS). However, the visual imaging quality inevitably suffers from several kinds of degradations (e.g., limited visibility, low contrast, color distortion, etc.) under complex weather conditions (e.g., haze, rain, and low-lightness). The degraded visual information will accordingly result in inaccurate environment perception and delayed operations for navigational risk. To promote the navigational safety of vessels, we propose a general-purpose multi-scene visibility enhancement method, the edge reparameterization- and attention-guided neural network (ERANet), to adaptively restore the degraded images captured under different weather conditions. In particular, our ERANet simultaneously exploits the channel attention, spatial attention, and reparameterization technology to enhance the visual quality while maintaining low computational cost. Extensive experiments conducted on standard and IWTS-related datasets have demonstrated that our ERANet could outperform several representative visibility enhancement methods in terms of both imaging quality and computational efficiency. The superior performance of IWTS-related object detection and scene segmentation could also be steadily obtained after ERANet-based visibility enhancement under complex weather conditions.

Introduction

Navigating the open seas is dangerous enough without the added challenge of "blindness" caused by weather. While visible-light cameras are standard on modern vessels for Intelligent Waterborne Transportation Systems (IWTS), their performance plummets during haze, heavy rain, or at night.

The core challenge isn't just "fixing" the image; it's doing it universally and instantly. Most existing computer vision models are "specific-purpose"—a model trained to remove rain often fails miserably when faced with a low-light sunset. Furthermore, marine vessels often rely on edge computing devices with limited processing power. A high-accuracy model is useless if it can't process video frames fast enough to prevent a collision.

Methodology

The proposed Edge Reparameterization- and Attention-guided Neural Network (ERANet) architecture rests on three pillars designed for efficiency and multi-scene robustness:

Kirsch-guided Reparameterization Module (KRM)

Instead of standard edge detection (like Sobel), ERANet uses Kirsch operators that calculate gradients in eight different directions. This is crucial for identifying complex rain streaks and obscured vessel edges. To maintain high inference speeds, the model uses structural reparameterization: during training, the model uses multiple complex branches to learn these edges, but for inference, these branches are mathematically collapsed into a single, efficient convolutional layer.

The Kirsch operator calculates gradients in eight directions to improve edge extraction while maintaining real-time performance.

Dual Attention Mechanism

The network employs both a Channel Attention Module (CAM) to fix color distortions across RGB channels and a Spatial Attention Module (SAM) to focus the processing power of the network on critical regions, like distant ships on the horizon.

Edge-guided Attention Residual Block (EARB)

This is the basic building block of the network, integrating the KRM and attention modules into a lightweight residual structure. The model is trained using a hybrid loss function that balances structural similarity (MS-SSIM), pixel-wise accuracy ($L_1$ loss), and spatial smoothness (Total Variation loss).

Experimental Results

ERANet was evaluated on a mix of general-purpose and maritime-specific datasets, including RESIDE-OTS (haze), Rain100L (rain), LOL (low-light), and the Singapore Maritime Dataset (SMD).

Efficiency and Quality

ERANet is incredibly fast, taking only 0.016 seconds to process a 1080p image (roughly 62 FPS). The model size is a mere 2.4 MB, making it ideal for the low-resource hardware found on autonomous surface vehicles. In quantitative tests (PSNR and SSIM), ERANet consistently outperformed state-of-the-art models like AirNet and TransWeather across all three weather conditions.

ERANet provides a unified solution for image dehazing, deraining, and low-light enhancement.

Downstream Synergy

Visibility enhancement is a critical preprocessing step. Clearing the "visual noise" leads to immediate gains in the accuracy of object detectors (like YOLOv7) and scene segmentors (like DeepLabv3+). When tested on the SMD dataset, ERANet improved the Mean Intersection over Union (mIoU) for scene segmentation from roughly 80-85% (with other methods) to 92.44%.

Conclusion

While ERANet is a powerhouse, it still struggles with extreme cases, such as excessively dark images or scenes with high local contrast. Future iterations will explore multi-scale features to handle varying rain streak thicknesses and move toward a Multi-Task Learning (MTL) framework where enhancement and object detection are handled simultaneously in a single, unified pipeline.

Summary updated

Abstract

The visible-light camera, which is capable of environment perception and navigation assistance, has emerged as an essential imaging sensor for marine surface vessels in intelligent waterborne transportation systems (IWTS). However, the visual imaging quality inevitably suffers from several kinds of degradations (e.g., limited visibility, low contrast, color distortion, etc.) under complex weather conditions (e.g., haze, rain, and low-lightness). The degraded visual information will accordingly result in inaccurate environment perception and delayed operations for navigational risk. To promote the navigational safety of vessels, many computational methods have been presented to perform visual quality enhancement under poor weather conditions. However, most of these methods are essentially specific-purpose implementation strategies, only available for one specific weather type. To overcome this limitation, we propose to develop a general-purpose multi-scene visibility enhancement method, i.e., edge reparameterization- and attention-guided neural network (ERANet), to adaptively restore the degraded images captured under different weather conditions. In particular, our ERANet simultaneously exploits the channel attention, spatial attention, and reparameterization technology to enhance the visual quality while maintaining low computational cost. Extensive experiments conducted on standard and IWTS-related datasets have demonstrated that our ERANet could outperform several representative visibility enhancement methods in terms of both imaging quality and computational efficiency. The superior performance of IWTS-related object detection and scene segmentation could also be steadily obtained after ERANet-based visibility enhancement under complex weather conditions.
BibTeX
@article{liu2024real,
  author = {Ryan Wen Liu and Yuxu Lu and Yuan Gao and Yu Guo and Wenqi Ren and Fenghua Zhu and Fei-Yue Wang},
  title = {{Real-Time Multi-Scene Visibility Enhancement for Promoting Navigational Safety of Vessels Under Complex Weather Conditions}},
  journal = {IEEE Trans. Intell. Transp. Syst.},
  year = {2024},
  doi = {10.1109/tits.2024.3454597},
}
Figure