Excavating RoI Attention for Underwater Object Detection

Xutao Liang, Pinhao Song

TL;DR

This paper introduces RoIAttn, a novel object detection method that enhances feature representation by applying an external attention mechanism to Region of Interest (RoI) features. By decoupling classification and regression tasks with a double-head structure and incorporating positional encoding for regression, RoIAttn achieves state-of-the-art performance, particularly in challenging underwater object detection scenarios.

What Problem Does This Paper Address?

Object detection, a fundamental task in computer vision, aims to identify and locate objects within an image. While Transformer architectures have revolutionized natural language processing and shown promise in computer vision, their application to object detection, especially in complex environments like underwater, still presents challenges. Existing methods often struggle with capturing intricate relationships between potential object regions (RoIs) and effectively distinguishing between classification and localization tasks. Underwater environments, in particular, introduce difficulties due to poor visibility, scattering, and color distortion, making robust object detection crucial for applications like marine research and autonomous underwater vehicles. This paper tackles the problem of improving object detection performance by leveraging attention mechanisms on RoI features, specifically addressing the need for more effective feature extraction and task-specific processing in challenging visual domains.

Methods

The core of RoIAttn lies in its novel RoI attention module. Instead of using computationally expensive self-attention directly on all pixels, it applies an external attention mechanism [9] to RoI features. External attention is a simplified form of self-attention that uses external memory units (key and value) to compute attention maps, significantly reducing computational complexity while still capturing relationships between RoIs. This module is designed to excavate inter-RoI relationships, enhancing the discriminative power of features.

To further improve performance, RoIAttn employs a double-head structure. This design separates the object detection task into two distinct branches: one for classification and one for regression. This separation acknowledges that different network components might be better suited for each task, with fully connected layers favored for classification and convolutional layers for regression.

Finally, to address the inherent difficulty of directly regressing precise coordinates, the paper introduces a positional encoding module. This module injects explicit spatial information into the features, particularly for the regression branch, by concatenating coordinate maps (Cx, Cy) with the base features and then downsampling them. This helps the model learn more accurate bounding box locations. The overall architecture builds upon a standard Faster R-CNN framework with a ResNet-50 backbone and FPN.

Datasets

The primary dataset used for evaluation and demonstrating the method's effectiveness is the UTDAC2020 dataset. This is an underwater object detection dataset specifically designed for the Underwater Target Detection Algorithm Competition 2020. It comprises 5168 training images and 1293 validation images, featuring four classes: echinus, holothurian, starfish, and scallop. The dataset presents challenges due to its high-resolution images and the inherent difficulties of underwater imaging.

The paper also reports results on the PASCAL VOC dataset to demonstrate the method's generalizability to more standard object detection benchmarks.

Key Insights and Ideas

  • RoI-level attention is powerful: The paper demonstrates that applying attention mechanisms not just at the pixel or patch level, but specifically to the extracted Region of Interest (RoI) features, can significantly boost object detection performance. This suggests that understanding the relationships between potential object regions is crucial.
  • External attention is an efficient alternative: For tasks with a potentially large number of RoIs, full self-attention can be computationally prohibitive. The adoption of external attention provides a practical and efficient way to leverage attention's benefits without excessive computational overhead.
  • Task-specific heads matter: Decoupling classification and regression into separate branches, each potentially optimized with different layer types (FC for classification, Conv for regression), leads to improved performance compared to a single-head approach.
  • Positional encoding aids regression: Explicitly injecting spatial information via positional encoding into the regression branch proves beneficial for achieving more accurate bounding box predictions, especially in scenarios where convolutional layers might struggle with direct coordinate regression.
  • Effectiveness in challenging domains: The method shows particular promise in the difficult underwater environment, highlighting its robustness to visual challenges.

Key Results

On the UTDAC2020 dataset, RoIAttn achieves a mean Average Precision (mAP) of 46.0%, outperforming the baseline Faster R-CNN by 1.5% and matching the performance of Libra R-CNN. The ablation studies show that the full RoIAttn model (baseline + RoI attention + double head + positional encoding) yields the best results. Specifically, applying RoI attention to the baseline increases mAP by 0.8%, and the double-head structure further boosts it to 45.8%. Adding positional encoding to the regression branch pushes the mAP to the final 46.0%.

On the PASCAL VOC dataset, RoIAttn achieves 81.1% mAP, an improvement of +1.6% over the baseline and +1.1% over Cascade R-CNN.

Limitations and Future Work

The paper notes that simply increasing the depth of the RoI attention module does not necessarily guarantee performance gains, suggesting that one attention module might be sufficient for convergence in tasks with a limited number of classes. The optimal hidden dimension d for the external memory units was found to be 10 for the UTDAC2020 dataset, which has only 4 classes, indicating that d might be related to the semantic complexity of the classes.

Future work could explore more advanced attention mechanisms or investigate how to further optimize the interaction between the RoI attention module and the backbone features. The authors also suggest that the proposed method could be extended to other dense prediction tasks beyond object detection.

Summary updated

Abstract

Self-attention is one of the most successful designs in deep learning, which calculates the similarity of different tokens and reconstructs the feature based on the attention matrix. Originally designed for NLP, self-attention is also popular in computer vision, and can be categorized into pixel-level attention and patch-level attention. In object detection, RoI features can be seen as patches from base feature maps. This paper aims to apply the attention module to RoI features to improve performance. Instead of employing an original self-attention module, we choose the external attention module, a modified self-attention with reduced parameters. With the proposed double head structure and the Positional Encoding module, our method can achieve promising performance in object detection. The comprehensive experiments show that it achieves promising performance, especially in the underwater object detection dataset. The code will be avaiable in: https://github.com/zsyasd/Excavating-RoI-Attention-for-Underwater-Object-Detection
BibTeX
@inproceedings{liang2022excavating,
  author = {Xutao Liang and Pinhao Song},
  title = {{Excavating RoI Attention for Underwater Object Detection}},
  booktitle = {ICIP},
  year = {2022},
  doi = {10.1109/icip46576.2022.9897515},
}
Figure