Balance Scene Learning Mechanism for Offshore and Inshore Ship Detection in SAR Images

Tianwen Zhang, Xiaoling Zhang, Jun Shi, Shunjun Wei, Jianguo Wang, Jianwei Li, Hao Su, Yue Zhou

TL;DR

The Balance Scene Learning Mechanism (BSLM) addresses the performance gap in SAR ship detection caused by the massive imbalance between plentiful "easy" offshore samples and scarce "complex" inshore samples. By using an unsupervised GAN-based clustering approach to identify and then augment inshore data, the authors significantly boost detection accuracy—improving inshore mean Average Precision (mAP) by up to 18.15%.

What Problem Does This Paper Address?

Synthetic Aperture Radar (SAR) is essential for maritime surveillance, but ship detection models often struggle in coastal (inshore) areas. The primary culprit is scene imbalance. In most datasets, offshore images (ships in open water) far outnumber inshore images (ships near docks or land).

Because offshore scenes are visually simple—bright ships against a dark, uniform background—deep learning models quickly master them. However, inshore scenes contain complex land clutter that mimics ship signatures, leading to high false-alarm rates and missed detections. Without a way to balance these scenes, models develop a "learning bias," over-optimizing for the easy offshore cases while failing in the high-stakes coastal environments.

Methods

The authors propose a three-step pipeline called BSLM that requires no manual scene labeling:

  1. Unsupervised Feature Extraction (GAN): Since most SAR datasets don't label images as "inshore" or "offshore," the authors use a Generative Adversarial Network. The GAN learns to represent the underlying distribution of SAR scenes. Specifically, they concatenate the feature maps from the last three layers of the GAN's discriminator to create a robust "scene feature vector."
  2. Scene Clustering (K-means): These feature vectors are fed into a K-means algorithm to group images into two clusters. Based on the statistical reality of SAR datasets, the smaller cluster is identified as "inshore" and the larger as "offshore."
  3. Balanced Augmentation: To eliminate the learning bias, the minority (inshore) samples are augmented using replication, rotation, or the addition of Gaussian noise until the two classes are balanced.

This balanced dataset is then used to train standard detectors like Faster R-CNN, Cascade R-CNN, SSD, and RetinaNet.

Datasets

The study utilizes the SSDD (SAR Ship Detection Dataset), the first open-source dataset of its kind. It contains 1,160 images from Sentinel-1, TerraSAR-X, and RadarSat-2 with various resolutions and polarizations. The authors adopted a refined 8:2 training/test split to ensure experimental consistency.

Key Insights and Ideas

  • The Power of Unsupervised Labeling: The most elegant part of the paper is using a GAN to "discover" the inshore/offshore distinction. This allows the method to be applied to any unlabeled SAR dataset without expensive manual annotation.
  • Addressing "Over-learning": The authors suggest that without BSLM, networks suffer from "over-learning" offshore samples. By forcing the network to spend more "attention" (via more training iterations on augmented data) on inshore samples, the model develops a more generalized representation of what a ship looks like across different backgrounds.
  • One-Stage vs. Two-Stage Sensitivity: One-stage detectors (SSD, RetinaNet) showed much larger performance gains from BSLM than two-stage detectors. This suggests that one-stage architectures are particularly vulnerable to background complexity and benefit more from balanced scene training.

Key Results

The BSLM consistently improved performance across all tested architectures:

  • Overall mAP Improvements: Faster R-CNN (+2.87%), Cascade R-CNN (+2.86%), SSD (+7.66%), and RetinaNet (+6.34%).
  • Inshore mAP Jumps: The most dramatic results were in complex scenes. For example, SSD’s inshore mAP increased by 18.15%, and RetinaNet’s by 12.40%.
  • Qualitative Success: Visual results show that BSLM-enabled models successfully suppressed false alarms on land and detected ships docked closely together that were previously missed.
  • Efficiency: Because BSLM is a training-time mechanism, it improves accuracy without reducing the inference speed of the final detector.

Limitations and Future Work

  • The Accuracy Gap: While BSLM significantly narrows the gap, inshore detection remains objectively harder than offshore detection. The "accuracy gap" still exists, albeit much smaller than before.
  • Augmentation Strategy: The paper notes that different detectors prefer different augmentation types. Two-stage detectors performed better with noise and rotation, while one-stage detectors were more sensitive to noise and performed better with simple replication.
  • Future Directions: The authors suggest that this universal mechanism could be integrated into other object detection tasks beyond ships and applied to even larger, more diverse SAR datasets.

Summary updated

Abstract

Huge imbalance of different scenes’ sample numbers seriously reduces synthetic aperture radar (SAR) ship detection accuracy. Thus, to solve this problem, this letter proposes a balance scene learning mechanism (BSLM) for offshore and inshore ship detection in SAR images. BSLM involves three steps: 1) based on unsupervised representation learning, a generative adversarial network (GAN) is used to extract the scene features of SAR images; 2) using these features, a scene binary cluster (offshore/inshore) is conducted by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${K}$ </tex-math></inline-formula> -means; and 3) finally, the small cluster’s samples (inshore) are augmented via replication, rotation transformation or noise addition to balance another big cluster (offshore), so as to eliminate scene learning bias and obtain balanced learning representation ability that can enhance learning benefits and improve detection accuracy. This letter applies BSLM to four widely used and open-sourced deep learning detectors, i.e., faster regions-convolutional neural network (Faster R-CNN), Cascade R-CNN, single shot multibox detector (SSD), and RetinaNet, to verify its effectiveness. Experimental results on the open SAR ship detection data set (SSDD) reveal that BSLM can greatly improve detection accuracy, especially for more complex inshore scenes.
BibTeX
@article{zhang2022balance,
  author = {Tianwen Zhang and Xiaoling Zhang and Jun Shi and Shunjun Wei and Jianguo Wang and Jianwei Li and Hao Su and Yue Zhou},
  title = {{Balance Scene Learning Mechanism for Offshore and Inshore Ship Detection in SAR Images}},
  journal = {IEEE Geosci. Remote. Sens. Lett.},
  year = {2022},
  url = {https://openreview.net/forum?id=3pkp4O4WWQ},
}

Loading related content...

Figure