Enhancing two-stage object detection models via data-driven anchor box optimization in UAV-based maritime SAR

Beigeng Zhao, Rui SongScientific 2024 · DOI

TL;DR

This paper tackles the challenge of accurately detecting objects in Unmanned Aerial Vehicle (UAV) maritime Search and Rescue (SAR) imagery. By analyzing the unique characteristics of SAR data and applying a data-driven anchor box optimization strategy using k-means clustering, the researchers significantly improved the performance of two-stage object detection models, particularly for challenging, small, and overlapping objects.

What Problem Does This Paper Address?

Maritime Search and Rescue (SAR) operations increasingly rely on Unmanned Aerial Vehicles (UAVs) for high-altitude surveillance. Accurately identifying critical objects like boats, swimmers, and life jackets in these images is paramount for effective rescue efforts. However, standard object detection models, often trained on general image datasets, struggle with the specific challenges of maritime SAR imagery. These challenges include significant variations in object scale, overlapping objects (e.g., a swimmer on a boat), and imbalanced class distributions (e.g., very few life jacket annotations). This paper aims to bridge this gap by optimizing object detection models specifically for this domain.

Methods

The core of this research lies in a data-driven approach to optimize anchor boxes for two-stage object detection models, specifically Faster R-CNN. The authors first conducted a thorough analysis of the SeaDronesSee dataset to understand its unique data characteristics and identify challenging object categories. Based on this analysis, they proposed an anchor box optimization strategy using k-means clustering.

Here's a breakdown of their approach:

  • Data Analysis: They analyzed object counts, sizes, and overlaps within the SeaDronesSee dataset.
  • Anchor Box Optimization:
    • K-means Clustering: They applied k-means clustering to the bounding box dimensions of objects in the dataset to identify optimal anchor box sizes. The elbow method was used to determine the optimal number of clusters (k=9).
  • FPN Layer-Specific Optimization: For models incorporating Feature Pyramid Networks (FPNs), they further refined anchor box optimization by applying it layer-by-layer to the FPN's multi-scale feature maps. This accounts for the different downscaling factors at each FPN level.
  • Model Selection: They experimented with Faster R-CNN models using different backbones (ResNet50, ResNet101, ResNeXt101 64x4d) and with/without FPN integration.

Datasets

The primary dataset used in this study is:

  • SeaDronesSee: A large-scale, open-water dataset specifically designed for UAV-based maritime SAR operations. It contains 5630 high-definition images with resolutions up to 5456x3632 pixels, annotated with ground truth bounding boxes following the COCO standard. This dataset is crucial as it captures the unique characteristics of the target domain, differing from general-purpose image datasets.

Key Insights and Ideas

  • Anchor Boxes Matter, Especially with FPNs: The study highlights that while anchor box optimization can offer improvements, its impact is significantly amplified when used in conjunction with Feature Pyramid Networks (FPNs). Models without FPNs either saw no improvement or even negative optimization.
  • Challenging Objects Benefit Most: The most substantial gains from anchor box optimization were observed in detecting small, overlapping objects like "swimmer on boat," "floater on boat," and "life jacket." These are precisely the categories that pose the greatest recognition challenges due to their size and occlusion.
  • Data-Driven is Key: The effectiveness of anchor box optimization is directly tied to the characteristics of the specific dataset. Generic anchor box configurations are less effective than those derived from clustering the actual object dimensions within the target domain.
  • FPN Layer-Specific Tuning is Crucial: Optimizing anchor boxes for each FPN layer individually, rather than using a single set of anchors for the entire model, leads to superior performance, especially for complex models.

Key Results

The paper reports significant improvements in object detection accuracy:

  • The proposed anchor box optimization method achieved a 45.8% increase in average precision over the default torchvision anchor box configuration and a 10% increase over the SeaDronesSee official sample code configuration when using FPN-enhanced models.
  • Specifically, the "F.ResNeXt101-FPN-FL" model (ResNeXt101 backbone, FPN, and layer-by-layer anchor optimization) demonstrated the best performance.
  • The improvements were most pronounced for the challenging categories:
  • "Swimmer on boat"
    • "Floater on boat"
    • "Life jacket"
  • While standard objects like "swimmer," "floater," and "boat" saw less dramatic improvements, the overall model performance was boosted by better detection of the difficult categories.

Limitations and Future Work

  • Limited Model Scope: The study focused on Faster R-CNN models. The authors acknowledge that validating their approach on other state-of-the-art two-stage models (e.g., Cascade R-CNN, RefineDet) would be beneficial.
  • Dataset Specificity: The optimization is data-driven and specific to the SeaDronesSee dataset. While promising, its direct applicability to other maritime SAR datasets or remote sensing scenarios would require further validation.
  • Test Set Discrepancy: A notable observation was a drop in model accuracy when applying predictions to the test set compared to the validation set. While the relative performance of different optimization strategies remained consistent, the underlying reasons for this generalization gap warrant further investigation.

Future work suggested includes extending the validation to a wider range of models and datasets, and further exploring the reasons behind the validation-test set performance discrepancies.

Summary updated

Abstract

The high-altitude imaging capabilities of Unmanned Aerial Vehicles (UAVs) offer an effective solution for maritime Search and Rescue (SAR) operations. In such missions, the accurate identification of boats, personnel, and objects within images is crucial. While object detection models trained on general image datasets can be directly applied to these tasks, their effectiveness is limited due to the unique challenges posed by the specific characteristics of maritime SAR scenarios. Addressing this challenge, our study leverages the large-scale benchmark dataset SeaDronesSee, specific to UAV-based maritime SAR, to analyze and explore the unique attributes of image data in this scenario. We identify the need for optimization in detecting specific categories of difficult-to-detect objects within this context. Building on this, an anchor box optimization strategy is proposed based on clustering analysis, aimed at enhancing the performance of the renowned two-stage object detection models in this specialized task. Experiments were conducted to validate the proposed anchor box optimization method and to explore the underlying reasons for its effectiveness. The experimental results show our optimization method achieved a 45.8% and a 10% increase in average precision over the default anchor box configurations of torchvision and the SeaDronesSee official sample code configuration respectively. This enhancement was particularly evident in the model's significantly improved ability to detect swimmers, floaters, and life jackets on boats within the SeaDronesSee dataset's SAR scenarios. The methods and findings of this study are anticipated to provide the UAV-based maritime SAR research community with valuable insights into data characteristics and model optimization, offering a meaningful reference for future research.
BibTeX
@inproceedings{zhao2024enhancing,
  author = {Beigeng Zhao and Rui Song},
  title = {{Enhancing two-stage object detection models via data-driven anchor box optimization in UAV-based maritime SAR}},
  booktitle = {Scientific},
  year = {2024},
  doi = {10.1038/s41598-024-55570-z},
}
Figure