Fine-grained Recognition of Ships Under Complex Sea Conditions

Jiaojiao Ma, Jun Yu, Haoqi Yang, Hong Jiang, Wei LiInternational Journal of Advanced Network Monitoring and Controls 2022 · DOI

TL;DR

This paper introduces an enhanced Faster R-CNN framework specifically designed to recognize ships in challenging maritime environments. By integrating guided filtering for defogging, multi-scale training for small targets, and negative sample enhancement, the authors achieved a 6.43% increase in precision and a 4.68% increase in recall over standard deep learning models.

What Problem Does This Paper Address?

Maritime target recognition is vital for both civilian safety and military surveillance. However, traditional deep learning models often struggle when deployed in real-world sea conditions. The authors identify three primary "blind spots" in current computer vision approaches for ship detection:

  1. Environmental Interference: Heavy fog and mist obscure the structural features of ships.
  2. Scale Variance: Ships in aerial or satellite imagery are often "small targets," occupying very few pixels, which makes feature extraction difficult.
  3. Background Noise: Coastal backgrounds and complex sea states create high "clutter," leading to frequent false positives where land features are mistaken for vessels.

Methods

The researchers built their solution on the Faster R-CNN architecture but introduced three critical modifications to handle the complexities of the sea:

  • Guided Filtering for Defogging: Instead of feeding "foggy" images directly into the network, the authors use an image processing technique called guided filtering. Based on the atmospheric scattering model, this method removes haze while specifically preserving the sharp edges of the ship's hull—crucial for the network to distinguish a ship from the water.
  • Multi-scale Training Strategy: To solve the small target problem, the team didn't just train on one image size. They resized training samples into three distinct scales (1024×1024, 512×512, and 256×256). This effectively "oversamples" smaller targets, forcing the network to learn robust features across different resolutions.
  • Negative Sample Enhancement: To reduce false alarms in coastal areas, the authors intentionally added 5,000 "negative" images—samples containing only fog or coastal land without ships—into the training set. This teaches the model what isn't a ship.
  • Backbone Selection: The authors opted for the ZF (Zeiler and Fergus) network as their feature extractor. They noted that while deeper networks like ResNet exist, the ZF network offered a better balance of structural simplicity and lower time complexity for this specific task.

Datasets

The study utilized a dataset provided by DataFountain, consisting of 33,397 images. The dataset includes four specific "fine-grained" classes of ships:

  • Huochuan (Cargo ships): 33,756 instances.
  • Youlun (Cruise ships): 8,028 instances.
  • Yuchuan (Fishing ships): 10,120 instances.
  • Youting (Yachts): 13,608 instances.

Additionally, 5,000 negative samples (foggy sea and coastal backgrounds) were integrated to improve the model's discrimination capabilities.

Key Insights and Ideas

  • Preprocessing Matters: In many CV tasks, researchers rely on the network to "learn" through the noise. This paper argues that for maritime fog, explicit mathematical defogging (Guided Filtering) is more effective than raw deep learning alone.
  • Scale Parity: By forcing the network to look at the same ship at multiple resolutions, the model becomes "scale-invariant." This prevents the common issue where a model recognizes a ship up close but fails when the drone or satellite is at a higher altitude.
  • The Power of "No": The inclusion of 5,000 negative samples highlights that teaching a model what a background looks like is just as important as teaching it what a target looks like, especially in high-interference environments like coastlines.

Key Results

The proposed model was compared directly against a standard Faster R-CNN implementation:

  • Precision: Increased from 88.13% to 94.56% (a 6.43% improvement).
  • Recall: Increased from 87.1% to 91.78% (a 4.68% improvement).
  • Efficiency: The authors report that the processing time per image is at the millisecond level, making it suitable for real-time monitoring applications.
  • Qualitative Success: Visual results demonstrated that the model could successfully identify ships in near-total "white-out" fog conditions and distinguish small yachts from complex coastal docks where standard models failed.

Limitations and Future Work

While the paper shows significant progress, a few areas remain open for exploration:

  • Backbone Constraints: The use of the ZF network, while efficient, might limit the model's ability to learn even more complex features compared to modern transformers or deeper residual networks.
  • Class Diversity: The model is trained on four specific ship types. Expanding this to more niche vessels (e.g., tugboats, naval vessels, or different types of cargo carriers) would be necessary for a truly universal maritime system.
  • Extreme Conditions: While "complex sea conditions" were addressed, the paper does not explicitly discuss performance in extreme night-time photography or heavy storm/rain conditions beyond standard fog.

Summary updated

Abstract

Abstract For the traditional deep learning cannot solve the fog, coastal background interference, and the difficulty of small ships recognition, a multi-scale deep learning training model is proposed in this paper. Based on Faster R-CNN, this paper uses guided filtering to remove fog, as well as combined with negative sample enhancement learning to train the model, thus solving recognition of ship in complex sea conditions. And with multi-scale training strategy, the multi-scale ship samples are produced and sent to the network for training, so as to solve the problem of small target recognition. The experimental results show that compared with the Faster R-CNN, the precision and recall of our method increase by 6.43% and by 4.68% respectively. It solves the difficulty of ships recognition under complex sea conditions and small ship recognition that cannot be solved by traditional deep learning methods, the trained model has good generalization ability and robustness.
BibTeX
@article{ma2022fine,
  author = {Jiaojiao Ma and Jun Yu and Haoqi Yang and Hong Jiang and Wei Li},
  title = {{Fine-grained Recognition of Ships Under Complex Sea Conditions}},
  journal = {International Journal of Advanced Network Monitoring and Controls},
  year = {2022},
  doi = {10.2478/ijanmc-2022-0035},
}
Figure