A Real-Time Ship Detector via a Common Camera

Penghui Zhao, Xiaoyuan Yu, Zongren Chen, Yanyan LiangJournal of Marine Science and Engineering 2022 · DOI

TL;DR

This paper introduces FRSD, a real-time ship detection system using common cameras that excels in challenging marine weather conditions like rain and fog. By combining a "fast U-Net" architecture with a novel "remapping attention" mechanism, FRSD achieves high accuracy (mAP > 83%) and speed, addressing the limitations of existing methods in adverse environments.

What Problem Does This Paper Address?

Detecting ships is crucial for maritime security and navigation. While advanced technologies like radar and satellites are effective for long-range monitoring, they are often too expensive for short-to-medium range applications. Common visible-light cameras offer rich visual information for distinguishing ship classes but struggle in adverse weather such as rain and fog. Existing ship detection models, often adapted from general object detection, lack robustness in these conditions and often fail to detect multiple ship classes in real-time. Furthermore, there's a significant gap in publicly available, diverse ship datasets and benchmarks specifically designed to test performance in challenging marine environments.

Methods

The core of the proposed FRSD system lies in two key innovations:

  • Fast U-Net: This is a modified U-Net architecture designed for efficiency. It compresses features in the channel dimension, reducing the number of training parameters and improving inference speed, particularly when processing multiple batches. This is achieved through indexing and slicing operations, which are more efficient than traditional upsampling methods like deconvolution.
  • Remapping Attention: This novel attention mechanism aims to enhance robustness in adverse weather. It works by remapping global spatial and channel features to local ones. This process helps the model focus on relevant information even when visual quality is degraded by rain or fog, thereby improving detection accuracy without sacrificing real-time performance.

The FRSD system builds upon the YOLOv5s architecture, leveraging its efficiency and then enhancing it with these new components.

Datasets

The authors introduce and utilize several datasets:

  • VOC-ship and VOC-ship-test: These are newly proposed ship datasets containing over 20,000 samples across eight specific ship classes and one "unidentified" class. They are designed to address the lack of diverse ship datasets and include various detection angles and challenging scenarios. Data augmentation, specifically "cross-background" augmentation, is used to further enhance background diversity.
  • Rain-fog Dataset: A new benchmark comprising over 500 pure rain-fog images. These images simulate various marine rain-fog scenarios and are used to quantitatively evaluate the robustness of ship detectors under different degrees of visual degradation.
  • COCO [25] and VOC [26]: These standard object detection datasets are used for initial training and validation of the remapping attention and fast U-Net components in more ideal conditions.

Key Insights and Ideas

  • Visible light cameras can be powerful for ship detection: Despite their susceptibility to weather, visible light cameras offer richer features than other sensors, making them a viable and economical option when combined with robust algorithms.
  • Adverse weather is a critical challenge: The performance of ship detectors degrades significantly in rain and fog, highlighting the need for specialized solutions.
  • Attention mechanisms can be tailored for specific challenges: The "remapping attention" specifically targets the problem of degraded visual input in marine environments, showing improved robustness compared to general attention methods.
  • Efficient architectures are key for real-time applications: The "fast U-Net" demonstrates that architectural modifications can significantly boost inference speed without a substantial drop in accuracy, making real-time detection feasible.
  • Dataset diversity is crucial for generalization: The creation of new, diverse ship and rain-fog datasets is essential for training and evaluating models that can perform reliably in real-world marine conditions.

Key Results

  • The proposed FRSD system achieves a mean Average Precision (mAP) of over 83% across 9 classes on their custom ship dataset.
  • FRSD demonstrates robustness in various rain-fog soaking conditions, with performance degradation being less severe compared to other detectors.
  • The fast U-Net component significantly accelerates inference speed, reaching up to 278.4 fps when multiple batches are used, while maintaining real-time performance (42.56 fps) even with a batch size of 1.
  • The remapping attention method shows a 3.5% increase in AP50 compared to a baseline U-Net on the COCO dataset, indicating its effectiveness.
  • The final model, FRSD_ship_dataaug, achieves the best performance across different soaking levels, with an mAP of 83.30% at Soaking 0.0 and 48.40% at Soaking 0.8.

Limitations and Future Work

The authors acknowledge several limitations:

  • Rain-fog Dataset Limitations: The proposed rain-fog dataset is primarily for validation and may not be sufficient for training models from scratch. The number of pure rain-fog scenes is also limited.
  • Adverse Weather Performance: While improved, the mean average precision still decreases significantly in severe rainy or foggy weather.
  • Real-time Speed vs. Robustness Trade-off: Enhancing anti-adverse weather capabilities might lead to a decrease in inference speed, posing a challenge for maintaining real-time performance.

Future work will focus on developing novel methods to further improve the anti-adverse weather capabilities of ship detectors without compromising real-time performance.

Summary updated

Abstract

Advanced radars and satellites, suitable for remote monitoring, inappropriately reach the economical requirements of short-range detection. Compared with far-sightedness skills, common visible-light sensors offer more ample features conducive to distinguishing the classes. Therefore, ship detection based on visible-light cameras should cooperate with remote detection technologies. However, compared with detectors applied in inland transportation, the lack of fast ship detectors, detecting multiple ship classes, is non-negligible. To fill this gap, we propose a real-time ship detector based on fast U-Net and remapping attention (FRSD) via a common camera. The fast U-Net offered compresses features in the channel dimension to decrease the number of training parameters. The remapping attention introduced boosts the performance in various rain–fog weather conditions while maintaining the real-time speed. The ship dataset proposed contains more than 20,000 samples, alleviating the lack of ship datasets containing various classes. Data augmentation of the cross-background is especially proposed to further promote the diversity of the detecting background. In addition, the rain–fog dataset proposed, containing more than 500 rain–fog images, simulates various marine rain–fog scenarios and soaks the testing image to validate the robustness of ship detectors. Experiments demonstrate that FRSD performs relatively robustly and detects 9 classes with an mAP of more than 83%, reaching a state-of-the-art level.
BibTeX
@article{zhao2022real,
  author = {Penghui Zhao and Xiaoyuan Yu and Zongren Chen and Yanyan Liang},
  title = {{A Real-Time Ship Detector via a Common Camera}},
  journal = {Journal of Marine Science and Engineering},
  year = {2022},
  doi = {10.3390/jmse10081043},
}
Figure