Marine Debris Detection in Satellite Surveillance Using Attention Mechanisms

Ao Shen, Yijie Zhu, Plamen Angelov, Richard Jiang

TL;DR

This paper tackles the challenge of detecting marine debris from satellite imagery by integrating YOLOv7 instance segmentation with various attention mechanisms. The study finds that the CBAM (Convolutional Block Attention Module) significantly improves detection accuracy, outperforming other attention models and the baseline YOLOv7.

What Problem Does This Paper Address?

Marine debris is a growing environmental concern, but current detection methods are often inefficient, labor-intensive, or limited in scope. While satellite imagery offers broad coverage, accurately identifying and segmenting small debris pieces from vast ocean surfaces remains a significant challenge. Existing deep learning approaches, while promising, often struggle with the resolution and detail required for effective marine debris detection from space. This paper aims to bridge this gap by enhancing a state-of-the-art object detection model (YOLOv7) with attention mechanisms to improve its ability to pinpoint and delineate marine debris in satellite images.

Methods

The core of this research involves augmenting the YOLOv7 instance segmentation model with different attention mechanisms. The authors experimented with three main types:

  • Coordinate Attention: This mechanism embeds location information into channel attention, allowing the model to capture both spatial dependencies and precise positional data.
  • CBAM (Convolutional Block Attention Module): This module combines both channel and spatial attention, aiming to focus on both the "what" and "where" of important features within an image.
  • Bottleneck Transformer (Self-Attention): This approach leverages self-attention, inspired by transformers, to allow the model to consider global relationships between features, potentially capturing long-range dependencies.

To improve the YOLOv7 architecture, a C2f module (a lightweight convolutional structure from YOLOv8) was also incorporated into some models. The authors then evaluated these modified models against the baseline YOLOv7 and YOLOv8.

Datasets

The study utilized satellite images from the PlanetScope mission, offering a resolution of 3 meters per pixel. These images were primarily from the Gulf Islands in Honduras and contained various types of debris, including plastic, wood, and algae. A dataset of 707 images was initially collected, but due to unclear imagery and cloud cover, 321 images were manually selected and labeled for training and testing. The dataset was split into a training set (79%) and a test set (21%).

Key Insights and Ideas

  • Attention is Key for Marine Debris: The experiments strongly suggest that incorporating attention mechanisms significantly boosts the performance of marine debris detection models, especially for instance segmentation.
  • CBAM Reigns Supreme: The CBAM module proved to be the most effective in improving both the precision and recall of marine debris detection, indicating its superior ability to focus on relevant features.
  • Bottleneck Transformer's Niche: While not the top performer overall, the bottleneck transformer showed promise in detecting debris missed by manual annotation and exhibited better mask precision for larger debris pieces, suggesting potential for specific use cases.
  • C2f Module's Limited Impact: The integration of the C2f module, while aiming for efficiency, did not consistently lead to performance improvements across all models, suggesting its effectiveness might be architecture-dependent.
  • Dataset Limitations are Critical: The scarcity of diverse, high-resolution satellite imagery specifically of ocean surfaces poses a significant challenge, impacting the robustness and generalizability of the models.

Key Results

The study evaluated models based on precision, recall, [email protected], and F1-score for both bounding box (BOX) and mask segmentation.

  • BOX Detection:
    • CBAM model achieved the highest F1 score of 0.77.
  • Coordinate Attention model scored 0.71.
  • Baseline YOLOv7 and Bottleneck Transformer models scored around 0.66.
  • MASK Segmentation:
    • CBAM model achieved the highest F1 score of 0.73.
  • Coordinate Attention model and YOLOv7 scored around 0.68-0.69.
  • Bottleneck Transformer model scored 0.56.
  • Precision: The self-attention model achieved the best BOX precision (0.832), while the CBAM model had the best MASK precision (0.787).
  • Recall: The Coordinate Attention model with C2f achieved the best recall for both BOX (0.787) and MASK (0.773).
  • Training Cost: The dual-attention model had a training cost about 25% longer than the self-attention model. CBAM model had twice the training time of other models (around 4 hours).

Limitations and Future Work

The authors acknowledge several limitations:

  • Dataset Scarcity and Bias: The dataset was limited in size and biased towards coastal areas, with a lack of diverse ocean surface imagery. Finding high-resolution satellite images exclusively of the open ocean proved difficult.
  • Homogeneous Dataset: The dataset's homogeneity limits the model's ability to generalize to various weather conditions, lighting, and debris types.
  • Small Sample Size: The limited number of samples hinders the model's capacity to discern subtle differences under varying environmental conditions.

Future work suggested includes:

  • Developing More Diverse Datasets: Acquiring high-quality satellite images from a wider range of environments, lighting conditions, and geographical regions is crucial.
  • Exploring Spectrograms: Using spectrograms instead of RGB images could potentially capture more detailed information for debris detection.
  • Integrating Oceanographic Knowledge: Incorporating sea current data could help forecast debris movement and improve the platform's value.
  • Advancing Deep Learning Techniques: Further research into self-attention mechanisms, particularly unsupervised learning approaches, could reduce reliance on manually labeled data and streamline the detection process.
  • Real-time Processing: Exploring ways to achieve true real-time detection, beyond just quick preliminary processing, is an area for further investigation.

Summary updated

Abstract

Marine debris poses a critical threat to environmental ecosystems, necessitating effective methods for its detection and localization. This study addresses the existing limitations in the literature by proposing an innovative approach that combines the instance segmentation capabilities of YOLOv7 with various attention mechanisms to enhance efficiency and broaden applicability. The primary contribution lies in the exploration and comparison of three attentional models: lightweight coordinate attention, combining spatial and channel focus (CBAM), and bottleneck transformer based on self-attention. Leveraging a meticulously labeled dataset of satellite images containing ocean debris, the study conducts a comprehensive assessment of box detection and mask evaluation. The results demonstrate that CBAM emerges as the standout performer, achieving the highest F1 score (77%) in box detection, surpassing coordinate attention (71%) and YOLOv7/bottleneck transformer (both around 66%). In mask evaluation, CBAM continues to lead with an F1 score of 73%, while coordinate attention and YOLOv7 exhibit comparable performances (around F1 scores of 68% and 69%), and bottleneck transformer lags behind at an F1 score of 56%. This compelling evidence underscores CBAM's superior suitability for detecting marine debris compared to existing methods. Notably, the study reveals an intriguing aspect of the bottleneck transformer, which, despite lower overall performance, successfully detected areas overlooked by manual annotation. Moreover, it demonstrated enhanced mask precision for larger debris pieces, hinting at potentially superior practical performance in certain scenarios. This nuanced finding underscores the importance of considering specific application requirements when selecting a detection model, as the bottleneck transformer may offer unique advantages in certain contexts.
BibTeX
@article{shen2024marine,
  author = {Ao Shen and Yijie Zhu and Plamen Angelov and Richard Jiang},
  title = {{Marine Debris Detection in Satellite Surveillance Using Attention Mechanisms}},
  journal = {IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens.},
  year = {2024},
  url = {https://openreview.net/forum?id=VFz5AhwV7s},
}
Figure