SSMA-YOLO: A Lightweight YOLO Model with Enhanced Feature Extraction and Fusion Capabilities for Drone-Aerial Ship Image Detection

Yuhang Han, Jizhuang Guo, Haoze Yang, Renxiang Guan, Tianjiao ZhangDrones 2024 · DOI

TL;DR

This paper introduces SSMA-YOLO, a lightweight YOLOv8n-based model for drone-aerial ship detection. By incorporating novel attention mechanisms (SSC2f, MC2f) and an improved feature fusion strategy (AFPN), SSMA-YOLO significantly boosts detection accuracy, especially for small and clustered ships in complex backgrounds, while simultaneously reducing model size.

What Problem Does This Paper Address?

Detecting ships from drone or satellite imagery presents unique challenges. Ships often appear as small objects with limited pixel representation, making them hard to distinguish from complex backgrounds. Furthermore, dense clusters of ships and varying environmental conditions (like waves and lighting) further degrade detection performance, leading to missed detections and false positives. Existing lightweight models, while fast, often struggle with these complexities, and more accurate models tend to be computationally heavy, hindering real-time applications crucial for maritime safety and resource management.

Methods

The SSMA-YOLO model builds upon the YOLOv8n architecture and introduces several key innovations:

  • SSC2f Module: This module replaces traditional convolutional blocks with a new structure that combines Spatial and Channel Convolution (SCConv) with Spatial Group-wise Enhancement (SGE) attention. This design aims to reduce spatial and channel redundancies, improving detection accuracy while making the model more lightweight.
  • MC2f Module: This module utilizes a Multidimensional Collaborative Attention (MCA) mechanism to better model spatial and channel features. It employs three parallel branches to capture dependencies across width, height, and channels, enhancing recognition in complex backgrounds.
  • AFPN Structure: To address the limitations of standard feature pyramid networks (FPN) in fusing multi-level features, the authors propose an Asymptotic Feature Pyramid Network (AFPN). This structure progressively fuses features, mitigating contradictions between different levels and improving the detection of multi-scale targets.
  • GhostConv: This lightweight convolution operation is used in the backbone to reduce computational load and parameter count.

Datasets

The primary dataset used for training and evaluation is the Ships dataset. This dataset consists of 9697 aerial and satellite images of ships, annotated in YOLO format. It includes images from various maritime areas, featuring a range of ship types and sizes. The paper also reports performance on two other datasets for generalization testing: SSDD (1160 high-resolution remote sensing images) and HRSC2016 (1170 remote sensing ship images).

Key Insights and Ideas

  • Attention is Key for Complexity: The paper strongly suggests that sophisticated attention mechanisms (like SGE and MCA) are crucial for disentangling ships from cluttered backgrounds and handling variations in ship appearance and scale.
  • Lightweight + Accurate is Achievable: By carefully designing modules like SSC2f and integrating GhostConv, the authors demonstrate that it's possible to achieve significant performance gains without a prohibitive increase in model complexity.
  • Progressive Feature Fusion Matters: The AFPN's approach to feature fusion, by gradually integrating information from different levels, proves more effective than standard methods for handling multi-scale targets.
  • Small Objects are Still a Challenge: Even with improvements, the paper implicitly highlights that detecting very small or heavily occluded ships remains an area for further research.

Key Results

The SSMA-YOLO model demonstrated superior performance compared to the baseline YOLOv8n:

  • mAP Improvement: Achieved a 4.4% increase in mean Average Precision (mAP) over YOLOv8n.
  • Parameter Reduction: Reduced the number of parameters by 23%.
  • Precision, Recall, F1: Showed improvements of 3.2%, 4.8%, and 4.2% in Precision, Recall, and F1 scores, respectively.
  • P-R Curve: Enclosed a larger Precision-Recall curve area (0.866 vs. 0.822 for YOLOv8n), indicating better overall performance.
  • Generalization: Performed well on the SSDD and HRSC2016 datasets, outperforming other mainstream object detection models.

Limitations and Future Work

The authors acknowledge a few limitations:

  • Dataset Coverage: The current dataset might not fully represent all complex scenarios, such as heavy fog or low-light conditions, potentially limiting the model's generalizability.
  • AFPN Complexity: While effective, the AFPN structure might require a deeper network, leading to increased computational complexity.
  • Attention Scope: The attention mechanisms primarily focus on spatial features from two directions, suggesting potential for improvement by considering more directions.

Future work will focus on:

  • Expanding Datasets: Gathering data from more extreme weather and diverse maritime environments to improve generalization.
  • Efficient Architectures: Exploring more compact network designs (e.g., depth-wise separable convolutions) to further reduce computational load.
  • Enhanced Attention: Investigating attention mechanisms that consider spatial features from more directions to improve accuracy.

Summary updated

Abstract

Due to the unique distance and angles involved in satellite remote sensing, ships appear with a small pixel area in images, leading to insufficient feature representation. This results in suboptimal performance in ship detection, including potential misses and false detections. Moreover, the complexity of backgrounds in remote sensing images of ships and the clustering of vessels also adversely affect the accuracy of ship detection. Therefore, this paper proposes an optimized model named SSMA-YOLO, based on YOLOv8n. First, this paper introduces a newly designed SSC2f structure that incorporates spatial and channel convolution (SCConv) and spatial group-wise enhancement (SGE) attention mechanisms. This design reduces spatial and channel redundancies within the neural network, enhancing detection accuracy while simultaneously reducing the model’s parameter count. Second, the newly designed MC2f structure employs the multidimensional collaborative attention (MCA) mechanism to efficiently model spatial and channel features, enhancing recognition efficiency in complex backgrounds. Additionally, the asymptotic feature pyramid network (AFPN) structure was designed for progressively fusing multi-level features from the backbone layers, overcoming challenges posed by multi-scale variations. Experiments of the ships dataset show that the proposed model achieved a 4.4% increase in mAP compared to the state-of-the-art single-stage target detection YOLOv8n model while also reducing the number of parameters by 23%.
BibTeX
@inproceedings{han2024ssma,
  author = {Yuhang Han and Jizhuang Guo and Haoze Yang and Renxiang Guan and Tianjiao Zhang},
  title = {{SSMA-YOLO: A Lightweight YOLO Model with Enhanced Feature Extraction and Fusion Capabilities for Drone-Aerial Ship Image Detection}},
  booktitle = {Drones},
  year = {2024},
  doi = {10.3390/drones8040145},
}
Figure