Multiscale Rotated Bounding Box-Based Deep Learning Method for Detecting Ship Targets in Remote Sensing Images

Shuxin Li, Zhilong Zhang, Biao Li, Chuwei Li2018 · DOI

TL;DR

This paper introduces a novel deep learning approach for detecting ships in remote sensing images, specifically addressing the challenges posed by arbitrary orientations and complex backgrounds. By employing multiscale rotated bounding boxes and a dual-path network, the method achieves more accurate localization and orientation estimation compared to traditional horizontal bounding box methods.

What Problem Does This Paper Address?

Detecting ships in remote sensing images presents several challenges. Firstly, ships can appear at any orientation due to the top-down perspective of satellite or aerial imagery. Using standard horizontal bounding boxes to enclose these oriented objects leads to significant background clutter within the box, making detection and precise localization difficult, especially when ships are close to each other or near the shore. Secondly, ships themselves vary in scale and appearance. Traditional manual methods are labor-intensive and lack robustness, while existing deep learning methods often still rely on horizontal bounding boxes, which are ill-suited for rotated targets. This paper aims to overcome these limitations by developing a deep learning method that can effectively detect and precisely locate ships, even in challenging environments with complex backgrounds and varying orientations.

Methods

The proposed method builds upon deep learning architectures, specifically adapting the principles of Faster R-CNN. The core innovation lies in the use of multiscale rotated bounding boxes for both detection and localization.

Here's a breakdown of the key methodological components:

  • Rotated Bounding Box Representation: Instead of the standard (x, y, w, h) for horizontal boxes, the authors use a five-parameter representation: (x_c, y_c, w, h, θ). Here, (x_c, y_c) is the center coordinate, w and h are the width and height (long and short sides, respectively), and θ is the angle of the long side with respect to the horizontal axis. This representation ensures the bounding box remains rectangular.
  • Deep Network Architecture: The network utilizes a VGG16 backbone, pre-trained on ImageNet, for feature extraction. This is a common practice in object detection.
  • Dual-Path Output: After feature extraction, the network branches into two parallel paths:
  • Classification Path: Predicts the class of the target (ship or background).
  • Regression Path: Predicts the location and angle offsets for the rotated bounding box.
  • Rotated Prior Boxes: Similar to anchor boxes in other object detection frameworks, the method uses pre-defined rotated prior boxes of various scales and orientations at each spatial location in the feature maps. These priors serve as initial guesses for potential ship locations and orientations.
  • Rotated Box Matching (Training): During training, a crucial step involves matching the rotated prior boxes with the ground-truth rotated bounding boxes. To reduce computational cost, a "range control strategy" is employed, narrowing down the search scope for matching. The matching degree is calculated based on the Intersection over Union (IoU) of the rotated boxes.
  • Loss Function: A combined loss function is used, comprising a classification loss (e.g., cross-entropy) and a regression loss (SmoothL1 loss) for both location and angle.
  • Non-Maximum Suppression (NMS): In the testing phase, NMS is applied to filter out redundant bounding box predictions and obtain the final detections.

Datasets

  • Custom Dataset: The authors collected 640 remote sensing images from Google Earth. This dataset contains 4297 ship targets with complex backgrounds, including wave clutter, ships in close proximity, and ships near the shore.
  • ImageNet: Used for pre-training the VGG16 backbone network.

Key Insights and Ideas

  • Rotated Boxes are Crucial for Orientation: The paper strongly demonstrates that for objects with arbitrary orientations like ships, rotated bounding boxes are fundamentally more suitable than horizontal ones. This directly addresses the issue of background clutter and improves localization accuracy.
  • Multiscale Detection is Key: Ships appear at various scales in remote sensing images. The use of multiscale prior boxes allows the model to effectively detect both small and large ships.
  • Angle Prediction is Feasible and Beneficial: Integrating angle prediction into the deep learning framework is shown to be effective, providing valuable orientation information beyond just location.
  • Range Control for Efficiency: The proposed "range control strategy" for matching prior boxes to ground-truth boxes is an important optimization that significantly reduces computational complexity during training without sacrificing accuracy.

Key Results

  • Improved AP: The proposed method achieved an Average Precision (AP) of 0.362 on their custom dataset. This is a significant improvement over:
  • SSD (horizontal boxes): 0.206 AP
    • Faster R-CNN (horizontal boxes): 0.293 AP
    • DRBox (rotated boxes): 0.305 AP (after resolution scaling)
    • OBB Faster R-CNN (rotated boxes): 0.278 AP
  • Robustness in Complex Conditions: Experiments showed the algorithm is robust in detecting ships under challenging conditions such as wave clutter, close proximity, and near the shore.
  • Better Small Ship Detection: The method demonstrated superior performance in detecting small ships compared to other rotated bounding box methods.
  • Precise Localization and Orientation: The algorithm not only detects ships but also provides accurate location and orientation information.

Limitations and Future Work

  • False Alarms: The authors acknowledge that false alarms, particularly on land backgrounds, still occur, indicating room for improvement in the model's precision.
  • Missing Detections: While improved, some targets, especially small ones, might still be missed.
  • Future Work: The authors explicitly state their intention to study the problems of target missing and false alarms in future work to further enhance the algorithm's performance.

Summary updated

BibTeX
@misc{li2018multiscale,
  author = {Shuxin Li and Zhilong Zhang and Biao Li and Chuwei Li},
  title = {{Multiscale Rotated Bounding Box-Based Deep Learning Method for Detecting Ship Targets in Remote Sensing Images}},
  year = {2018},
  doi = {10.3390/s18082702},
}
Figure