TL;DR
This paper introduces the Broad Area Target Search (BATS) system, a comprehensive framework for ship detection in high-resolution remote sensing imagery. BATS integrates two novel deep learning approaches: Saliency-Faster R-CNN to address multi-scale ship detection challenges and Mask-Faster R-CNN to mitigate onshore false alarms, achieving improved accuracy and robustness.
What Problem Does This Paper Address?
Detecting ships in high-resolution remote sensing images is crucial for maritime domain awareness, but it's fraught with challenges. These include:
- Multi-scale ships: Ships appear at vastly different sizes in images, and standard deep learning models often struggle with this variability, especially when training data is dominated by smaller ship examples. This leads to missed detections of larger vessels.
- Onshore false alarms: The presence of land, harbors, and other coastal structures can be misidentified as ships by detection algorithms, particularly in near-shore areas. This "onshore false alarm" problem significantly degrades detection accuracy.
- Environmental factors: While not the primary focus of the proposed methods, the paper acknowledges that factors like clouds and waves can also impact detection.
The authors aim to build a robust system that can effectively handle these issues in broad-area surveillance.
Methods
The core of the BATS system lies in two novel DCNN-based methods:
- Saliency-Faster R-CNN: This approach tackles the multi-scale ship detection problem. It first uses a saliency estimation network to identify regions of interest and distinguish between scenes with large-scale and small-scale ships. Then, an image pyramid is employed to compress images containing large ships, making their scale more consistent with the training data. This allows the object detection network (based on Faster R-CNN) to better learn and detect ships of varying sizes.
- Mask-Faster R-CNN: This method addresses onshore false alarms. It integrates a scene mask extraction network (SMEN) into the Faster R-CNN architecture. SMEN performs pixel-level classification to differentiate between sea (target) and land (non-target) areas. This mask is then used to suppress features from non-target (land) regions in the feature maps before the region proposal network (RPN) processes them, effectively filtering out potential false alarms from land areas.
The BATS system itself is a framework that orchestrates these detection modules with data import, pre-processing, results display, and a manual review interface for iterative improvement.
Datasets
- Training Dataset: DOTA (A large-scale dataset for object detection in aerial images). The authors selected 400 images from DOTA, specifically focusing on those with multi-scale ships and near-harbor scenes, and manually labeled mask information for training.
- Testing Dataset: A combination of images randomly picked from the DOTA testing set and remote sensing images collected from Google Earth (with a resolution of 0.3 m).
No new benchmark datasets were introduced.
Key Insights and Ideas
- Saliency for Scale Awareness: Using saliency estimation to guide the scale adjustment process is a clever way to adapt the detection pipeline to varying ship sizes, especially when training data is imbalanced.
- Semantic Segmentation for False Alarm Suppression: Integrating a semantic segmentation branch (SMEN) directly into the object detection pipeline is an effective strategy to explicitly delineate target (sea) from non-target (land) areas, directly combating onshore false alarms.
- Cascading Models for Comprehensive Performance: Combining the Saliency-Faster R-CNN and Mask-Faster R-CNN within the BATS system demonstrates that a multi-pronged approach can yield superior results by addressing distinct challenges simultaneously.
- Data-Driven System Design: The BATS system's emphasis on a user interface for manual review and iterative optimization highlights the importance of human-in-the-loop learning for refining deep learning models in complex real-world applications.
Key Results
- Saliency-Faster R-CNN: Achieved an Average Precision (AP) of 0.727 on large-scale ships (APL), significantly outperforming the baseline Faster R-CNN (0.664). The Non-Compressing Rate (NCR) was low (0.105), indicating effective handling of large-scale ships.
- Mask-Faster R-CNN: Showed a 2.2% higher AP (0.628 vs. 0.606) compared to Faster R-CNN. Crucially, it reduced the false alarm rate (False Rate) from 0.686 to 0.397 and achieved a high mean Intersection over Union (mIOU) of 0.877 for scene mask estimation.
- BATS System (Cascading Model): The combined model achieved an AP of 0.629, showing a slight improvement over Mask-Faster R-CNN alone. While the inference time increased (2.233 sec/image) due to the saliency estimation, it demonstrated a good balance between accuracy and detection of large-scale ships.
Limitations and Future Work
- Inference Time: The proposed methods, especially the cascading model and Saliency-Faster R-CNN, incur higher inference times (e.g., 2.181 sec/image for Saliency-Faster R-CNN) compared to the baseline Faster R-CNN (0.173 sec/image). This is noted as a potential drawback for real-time detection tasks with massive datasets.
- Generalization to Other Targets: The current BATS system is specialized for ship detection. Future work aims to extend its capabilities to detect other targets like airplanes and vehicles.
- All-Weather Capability: The authors suggest incorporating Synthetic Aperture Radar (SAR) images as a complement to optical imagery to achieve all-weather detection, especially in conditions with heavy clouds and fog.
- Model Optimization: Continuous optimization of the DCNN models is planned to adapt to the diversity of targets and scenes.
Summary updated