Attention-Based Multiscale Feature Pyramid Network for Corn Pest Detection under Wild Environment

Chenrui Kang, Lin Jiao, Rujing Wang, Zhigui Liu, Jianming Du, Haiying HuInsects 2022 · DOI

TL;DR

This paper introduces a novel deep learning model, the Attention-Based Multiscale Feature Pyramid Network (AMFFP-Net), for accurately detecting corn pests in challenging wild environments. By combining deformable convolutions for robust feature extraction and an attention mechanism within a feature pyramid network, the model achieves state-of-the-art performance on a newly created large-scale corn pest dataset, balancing high accuracy with efficient processing speeds.

What Problem Does This Paper Address?

Corn is a vital crop, but its production is significantly threatened by pest outbreaks. Traditional methods for monitoring corn pests rely on manual observation, which is labor-intensive, time-consuming, and often provides lagging information, especially in remote areas. This makes timely prevention and control difficult, leading to substantial crop losses. The challenge lies in developing an automated, accurate, and efficient system for detecting corn pests in real-world, "wild" environments, which are characterized by complex backgrounds, varying lighting conditions, and diverse pest appearances (size, pose, occlusion). Existing automated methods often struggle with these complexities, necessitating a more robust approach.

Methods

The proposed method integrates several key deep learning components:

  • Deep Residual Network with Deformable Convolution: To enhance feature extraction, the authors incorporate deformable convolutions into a deep residual network. Unlike standard convolutions that use fixed sampling grids, deformable convolutions learn offsets to adapt their sampling locations. This allows the network to better capture the geometric variations and deformations inherent in pest appearances, improving robustness against pose and viewpoint changes.
  • Attention-Based Multi-Scale Feature Fusion Pyramid Network (AMFFP-Net): To effectively detect pests of varying sizes, the paper builds upon the Feature Pyramid Network (FPN) concept. FPNs combine low-level and high-level feature maps to create a pyramid of features at different scales. The novelty here is the introduction of an attention mechanism within the fusion process. This attention mechanism learns to assign different weights to features from different layers, allowing the network to better distinguish between features and improve the overall representation of corn pests, especially when they appear at different scales.
  • Two-Stage Detector Integration: The developed deep residual network and AMFFP-Net are integrated into a Faster R-CNN framework, a well-established two-stage object detection architecture. This combination allows for both the identification (classification) and localization (bounding box prediction) of corn pests within an image.

Datasets

  • Newly Constructed Large-Scale Corn Pest Dataset: The paper introduces a new dataset comprising 7392 corn pest images across 10 different pest classes. These images were collected in the wild using mobile phones and cameras between 2018 and 2020 from the Anhui and Henan provinces in China. The dataset is notable for its real-world conditions, including variations in size, illumination, and occlusion.
  • Data Splitting: The dataset is split into a training set (90%) and a testing set (10%), with 6653 images for training and 740 for testing.

Key Insights and Ideas

  • Deformable Convolutions for Pest Robustness: The use of deformable convolutions is a crucial insight for handling the inherent variability in pest appearance. By allowing the network to adapt its receptive field, it can better focus on relevant pest features despite changes in pose or viewpoint, which is a significant advantage in wild environments.
  • Attention for Multi-Scale Feature Fusion: The attention mechanism in the AMFFP-Net is key to improving the fusion of features from different scales. It allows the network to dynamically weigh the importance of features, leading to a more discriminative representation for detecting pests of various sizes, a common challenge in object detection.
  • Dataset Importance: The creation of a large-scale, diverse dataset of corn pests in wild environments is a significant contribution. Such datasets are essential for training and evaluating robust deep learning models for agricultural applications.

Key Results

  • High Accuracy: The proposed method achieves a mean Average Precision (mAP) of 70.1% and a mean Recall of 74.3% on the corn pest dataset. This represents an improvement over baseline methods like FPN, S-RPN, and Cascade R-CNN.
  • Efficient Performance: The model operates at a speed of 17.0 frames per second (FPS) on a single NVIDIA GPU, demonstrating a good balance between accuracy and efficiency for practical applications.
  • Ablation Study: Deformable convolution alone improved mAP from 65.2% to 66.3%, while the AMFFP-Net module further boosted performance by 3.8% mAP, highlighting the effectiveness of both components.

Limitations and Future Work

  • Occlusion and Dense Distribution: The authors acknowledge that severe occlusion and dense distribution of pests remain challenges. Figure 4 illustrates examples of occlusion, and Figure 6 shows instances where pests are undetected due to these issues.
  • Future Directions: The authors plan to address these limitations in future work by adopting a "coarse-to-fine" strategy to better handle dense distributions and overlapping pests.

Summary updated

Abstract

A serious outbreak of agricultural pests results in a great loss of corn production. Therefore, accurate and robust corn pest detection is important during the early warning, which can achieve the prevention of the damage caused by corn pests. To obtain an accurate detection of corn pests, a new method based on a convolutional neural network is introduced in this paper. Firstly, a large-scale corn pest dataset has been constructed which includes 7741 corn pest images with 10 classes. Secondly, a deep residual network with deformable convolution has been introduced to obtain the features of the corn pest images. To address the detection task of multi-scale corn pests, an attention-based multi-scale feature pyramid network has been developed. Finally, we combined the proposed modules with a two-stage detector into a single network, which achieves the identification and localization of corn pests in an image. Experimental results on the corn pest dataset demonstrate that the proposed method has good performance compared with other methods. Specifically, the proposed method achieves 70.1% mean Average Precision (mAP) and 74.3% Recall at the speed of 17.0 frames per second (FPS), which balances the accuracy and efficiency.
BibTeX
@inproceedings{kang2022attention,
  author = {Chenrui Kang and Lin Jiao and Rujing Wang and Zhigui Liu and Jianming Du and Haiying Hu},
  title = {{Attention-Based Multiscale Feature Pyramid Network for Corn Pest Detection under Wild Environment}},
  booktitle = {Insects},
  year = {2022},
  doi = {10.3390/insects13110978},
}
Figure