RSI-YOLO: Object Detection Method for Remote Sensing Images Based on Improved YOLO

Zhuang Li, Yuan Jian-hui, Guixiang Li, Hao Wang, Xingcan Li, Dan Li, Xinhua WangSensors 2023 · DOI

TL;DR

This paper introduces RSI-YOLO, an enhanced YOLOv5 model for detecting small objects in remote sensing images. By incorporating attention mechanisms, a weighted Bi-FPN for feature fusion, an additional detection layer for small objects, and an improved EIOU loss function, RSI-YOLO significantly boosts detection accuracy and robustness compared to the original YOLOv5 and other state-of-the-art methods.

What Problem Does This Paper Address?

Remote sensing images are crucial for various applications like environmental monitoring and urban planning. However, detecting objects within these images presents a significant challenge due to the objects' small size and low resolution, often making them difficult to discern. Traditional methods struggle with the complexity and diversity of remote sensing data. While deep learning, particularly object detection algorithms like YOLO, has shown promise, existing models often face limitations in accurately identifying and localizing these small, often densely packed, targets without sacrificing speed. This paper aims to bridge this gap by improving YOLOv5 to specifically address the challenges of small object detection in remote sensing contexts.

Methods

The researchers built upon the YOLOv5 architecture, a popular one-stage object detector known for its balance of speed and accuracy. Their improvements are multi-faceted:

  • Attention Mechanisms: They integrated the Convolutional Block Attention Module (CBAM), which combines channel and spatial attention. This allows the network to focus on more relevant features and suppress irrelevant information, mimicking human visual attention.
  • Enhanced Feature Fusion: The original PANet structure in YOLOv5's neck was replaced with a weighted Bidirectional Feature Pyramid Network (Bi-FPN). This modification aims for more efficient and richer fusion of multi-scale features, crucial for capturing information from objects of varying sizes.
  • Small Object Detection Layer: An additional detection layer was introduced. This is specifically designed to improve the network's ability to detect very small objects, which can be easily missed by standard detection layers due to downsampling or limited receptive fields.
  • Improved Loss Function: The standard CIOU loss was replaced with the Efficient Intersection over Union (EIOU) loss. EIOU directly optimizes the width and height of bounding boxes, leading to faster convergence and better regression accuracy, especially for small objects.

Datasets

The proposed RSI-YOLO model was trained and evaluated on several benchmark datasets:

  • DOTA: A large-scale dataset for object detection in aerial images, containing 15 categories. The original images were cropped to 1024x1024 for this study.
  • DIOR: Another dataset for object detection in aerial images, featuring 20 categories and a larger number of targets. Images were resized to 800x800.
  • NWPU VHR-10: A smaller dataset with 10 categories, used to validate algorithmic improvements.
  • MAR20: A specialized dataset focusing on military aircraft recognition, providing fine-grained model information for 20 types of airplanes.

Key Insights and Ideas

  • Attention is Key for Small Objects: The integration of CBAM highlights that focusing on salient features, both spatially and channel-wise, is critical for distinguishing small objects from complex backgrounds in remote sensing imagery.
  • Bi-FPN for Richer Features: Replacing the standard PANet with a weighted Bi-FPN demonstrates that intelligently fusing features from different scales, with learned weights, leads to a more robust representation, benefiting the detection of objects across a wide size spectrum.
  • Dedicated Small Object Layer Pays Off: The addition of a specific detection layer for small objects directly addresses a known weakness in many object detectors, showing that specialized architectural components can yield significant improvements.
  • EIOU for Better Convergence: The shift to EIOU loss is not just a minor tweak; it directly impacts the bounding box regression, enabling faster training and more accurate localization, particularly for challenging small targets.

Key Results

The RSI-YOLO model demonstrated superior performance across multiple datasets:

  • DOTA Dataset: Achieved AP0.55 of 45.3% and AP0.5 of 71.2%. Compared to YOLOv5s, RSI-YOLO showed improvements in recall (4.6% overall) and AP metrics (1.3% for AP0.5, 1.6% for AP0.55). Qualitative results showed RSI-YOLO successfully detected targets missed by YOLOv5s, such as airplanes and harbors.
  • DIOR Dataset: Achieved AP0.55 of 79.4% and AP0.5 of 57.2%. The model showed improved robustness, correctly identifying targets that YOLOv5s missed or misclassified.
  • NWPU VHR-10 Dataset: Achieved AP0.55 of 95.8% and AP0.5 of 57.9%. RSI-YOLO outperformed Faster R-CNN and SSD in several categories, particularly in AP0.5.
  • MAR20 Dataset: RSI-YOLO showed improvements in AP0.95 over YOLOv5s, with specific categories like "A2", "A9", and "A20" showing notable increases.
  • Ablation Studies: The study systematically evaluated the impact of each improvement. Combining all enhancements (CBAM, Bi-FPN, small object layer, EIOU) resulted in the highest performance gains, with AP0.5 increasing by 1.3% and AP0.55 by 1.6% over the baseline YOLOv5s.

Limitations and Future Work

The authors acknowledge that while RSI-YOLO improves accuracy, the addition of modules and detection layers increases the model's parameter count and FLOPs, leading to higher computational complexity and potentially slower inference speeds. This could be a constraint in resource-limited engineering applications.

Future research directions suggested include:

  • Further improving average precision: Exploring methods to further enhance performance across various datasets.
  • Reducing model complexity: Investigating techniques like model pruning and knowledge distillation to decrease model size and speed up detection without sacrificing accuracy, making it more suitable for deployment.
  • Data augmentation: Utilizing additional datasets and data augmentation strategies to enrich the training set and improve generalization.

Summary updated

Abstract

With the continuous development of deep learning technology, object detection has received extensive attention across various computer fields as a fundamental task of computational vision. Effective detection of objects in remote sensing images is a key challenge, owing to their small size and low resolution. In this study, a remote sensing image detection (RSI-YOLO) approach based on the YOLOv5 target detection algorithm is proposed, which has been proven to be one of the most representative and effective algorithms for this task. The channel attention and spatial attention mechanisms are used to strengthen the features fused by the neural network. The multi-scale feature fusion structure of the original network based on a PANet structure is improved to a weighted bidirectional feature pyramid structure to achieve more efficient and richer feature fusion. In addition, a small object detection layer is added, and the loss function is modified to optimise the network model. The experimental results from four remote sensing image datasets, such as DOTA and NWPU-VHR 10, indicate that RSI-YOLO outperforms the original YOLO in terms of detection performance. The proposed RSI-YOLO algorithm demonstrated superior detection performance compared to other classical object detection algorithms, thus validating the effectiveness of the improvements introduced into the YOLOv5 algorithm.
BibTeX
@inproceedings{li2023rsi,
  author = {Zhuang Li and Yuan Jian-hui and Guixiang Li and Hao Wang and Xingcan Li and Dan Li and Xinhua Wang},
  title = {{RSI-YOLO: Object Detection Method for Remote Sensing Images Based on Improved YOLO}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23146414},
}
Figure