Position Detection and Direction Prediction for Arbitrary-Oriented Ships via Multitask Rotation Region Convolutional Neural Network

Xue Yang, Hao Sun, Xian Sun, Menglong Yan, Zhi Guo, Kun Fu

TL;DR

This paper introduces a specialized deep learning framework for detecting ships in high-resolution satellite imagery using rotational bounding boxes instead of standard horizontal ones. By incorporating a multiscale feature network and a novel "prow direction" prediction branch, the model can accurately locate densely packed ships and determine their sailing or berthing direction, significantly outperforming traditional detection methods.

What Problem Does This Paper Address?

Detecting ships in remote sensing (satellite) imagery is notoriously difficult for two main reasons:

  1. Large Aspect Ratios: Ships are long and thin. When they are oriented diagonally, a standard horizontal bounding box includes a massive amount of "noise" (background water or docks), which confuses the neural network.
  2. Dense Packing: In ports, ships are often parked side-by-side. Horizontal boxes for adjacent ships overlap so much that standard Non-Maximum Suppression (NMS) algorithms—which are designed to remove duplicate detections—often accidentally delete valid ships, thinking they are overlaps of the same object.

Furthermore, most detectors only tell you where a ship is, not which way it is pointing, which is critical for maritime traffic monitoring.

Methods

The authors propose a Multiscale Rotation Region Convolutional Neural Network consisting of five key components:

  • Dense Feature Pyramid Network (DFPN): Building on standard FPNs, this architecture uses dense connections to fuse low-level positional information with high-level semantic features. This ensures the model can "see" both tiny boats and massive tankers effectively.
  • Adaptive ROI Align: Standard ROI Align (from Mask R-CNN) includes too much background noise for tilted ships. Conversely, "Rotational" ROI Align can lose spatial information. The authors created an adaptive version that uses a trainable mask to filter out noise while preserving the ship's spatial integrity.
  • Rotational Bounding Box Regression: Instead of just $(x, y, width, height)$, the model predicts a fifth variable: the angle ($\theta$). This allows the box to tilt and tightly hug the ship's hull.
  • Prow Direction Prediction: The model treats the four sides of the bounding box as potential "heads." A specific branch of the network predicts which side is the prow (front), allowing the system to determine the ship's orientation.
  • Rotational NMS (R-NMS): A modified suppression algorithm that calculates the Intersection-over-Union (IoU) of tilted rectangles, preventing the accidental deletion of ships parked closely together.

Datasets

The model was evaluated on two primary datasets:

  1. SRSS (Satellite Remote Sensing Ship): A dataset collected and labeled by the authors using Google Earth. It contains 50 large scenes (10,000 x 10,000 pixels) at 0.5m resolution, sliced into ~8,000 sub-images.
  2. DOTA: A large-scale, publicly available dataset for object detection in aerial images. It contains 15 categories (including ships, planes, and vehicles) and is the gold standard for testing "arbitrary-oriented" object detection.

Key Insights and Ideas

  • The "Prow" Logic: The authors observed that the network naturally learns that the prow must be on one of the "short" sides of the long rectangular box. By adding a simple classification layer to choose between the two short sides, they achieved high direction-prediction accuracy without massive computational overhead.
  • Noise vs. Geometry: The "Adaptive ROI Align" is a clever compromise. It acknowledges that while we want to exclude background pixels (noise), forcing a perfect geometric rotation (RROI) can actually "break" the features the network has learned to recognize. The mask-based approach allows the network to learn what to ignore.
  • Density is the Real Test: The paper highlights that rotational detection isn't just about looking "cleaner"—it's a functional necessity for ports. Without rotation, the detector is mathematically blind to ships parked side-by-side.

Key Results

  • DOTA Performance: The model achieved a Mean Average Precision (mAP) of 62.29, significantly higher than other popular models like SSD (17.84), YOLOv2 (25.49), and even the standard Faster R-CNN (39.95).
  • Ship-Specific Accuracy: On the SRSS dataset, the model reached an F1-score of 84.9%, outperforming multiscale FPNs and LSTM-based detectors.
  • Directional Accuracy: The prow prediction accuracy closely tracked the overall recall rate, proving that if the model can find the ship, it can almost always tell which way it's pointing.

Limitations and Future Work

The authors acknowledge that while the model is highly accurate, it still suffers from false alarms in complex environments (e.g., mistaking certain dock structures for ships). Future work will focus on refining the feature extraction to better distinguish between man-made coastal structures and actual vessels. Additionally, while the model is efficient, it is slightly slower than "single-shot" detectors like YOLO, suggesting a need for speed optimization for real-time applications.

Summary updated

Abstract

Ship detection is of great importance and full of challenges in the field of remote sensing. The complexity of application scenarios, the redundancy of detection region, and the difficulty of dense ship detection are all the main obstacles that limit the successful operation of traditional methods in ship detection. In this paper, we propose a brand new detection model based on multitask rotational region convolutional neural network to solve the problems above. This model is mainly consisting of five consecutive parts: dense feature pyramid network, adaptive region of interest (ROI) align, rotational bounding box regression, prow direction prediction and rotational nonmaximum suppression (R-NMS). First of all, the low-level location information and high-level semantic information are fully utilized through multiscale feature networks. Then, we design adaptive ROI align to obtain high quality proposals which remain complete spatial and semantic information. Unlike most previous approaches, the prediction obtained by our method is the minimum bounding rectangle of the object with less redundant regions. Therefore, the rotational region detection framework is more suitable to detect the dense object than traditional detection model. Additionally, we can find the berthing and sailing direction of ship through prediction. A detailed evaluation based on SRSS for rotation detection shows that our detection method has a competitive performance.
BibTeX
@inproceedings{yang2018position,
  author = {Xue Yang and Hao Sun and Xian Sun and Menglong Yan and Zhi Guo and Kun Fu},
  title = {{Position Detection and Direction Prediction for Arbitrary-Oriented Ships via Multitask Rotation Region Convolutional Neural Network}},
  booktitle = {IEEE Access},
  year = {2018},
  url = {https://openreview.net/forum?id=-n__F0oxOMn},
}

Loading related content...

Figure