Accelerating Image-based Pest Detection on a Heterogeneous Multi-core Microcontroller

Luca Bompani, Luca Crupi, Daniele Palossi, Olmo Baldoni, Davide Brunelli, Francesco Conti, Manuele Rusci, Luca Benini

Abstract

The codling moth pest poses a significant threat to global crop production, with potential losses of up to 80% in apple orchards. Special camera-based sensor nodes are deployed in the field to record and transmit images of trapped insects to monitor the presence of the pest. This paper investigates the embedding of computer vision algorithms in the sensor node using a novel State-of-the-Art Microcontroller Unit (MCU), the GreenWaves Technologies' GAP9 System-on-Chip, which combines 10 RISC-V general purposes cores with a convolution hardware accelerator. We compare the performance of a lightweight Viola-Jones detector algorithm with a Convolutional Neural Network (CNN), MobileNetV3-SSDLite, trained for the pest detection task. On two datasets that differentiate for the distance between the camera sensor and the pest targets, the CNN generalizes better than the other method and achieves a detection accuracy between 83% and 72%. Thanks to the GAP9's CNN accelerator, the CNN inference task takes only 147 ms to process a 320x240 image. Compared to the GAP8 MCU, which only relies on general-purpose cores for processing, we achieved 9.5x faster inference speed. When running on a 1000 mAh battery at 3.7 V, the estimated lifetime is approximately 199 days, processing an image every 30 seconds. Our study demonstrates that the novel heterogeneous MCU can perform end-to-end CNN inference with an energy consumption of just 4.85 mJ, matching the efficiency of the simpler Viola-Jones algorithm and offering power consumption up to 15x lower than previous methods.

Introduction

The codling moth (Cydia pomonella) is a significant threat to global apple production, often referred to as the "billion-dollar bug" due to its potential to destroy up to 80% of an orchard's yield. Traditional monitoring involves manual inspection of pheromone traps, which is labor-intensive and slow. Modern automated solutions utilize camera-based sensor nodes to capture and transmit images of trapped insects. However, transmitting high-resolution images over cellular networks is energy-intensive, accounting for roughly 66% of a node's total energy consumption. Near-sensor processing—analyzing images locally and only transmitting counts—offers a solution, but has historically been limited by the low computational power of standard microcontrollers (MCUs).

Automated pheromone trap (A) and captured codling moths on a sticky pad (B).

Methodology and Hardware Architecture

This study evaluates the deployment of advanced computer vision algorithms on the GreenWaves Technologies’ GAP9 System-on-Chip (SoC). The GAP9 is a heterogeneous MCU featuring 10 RISC-V cores and a dedicated convolution hardware accelerator known as the NE16.

The authors compared two distinct approaches:

  1. Viola-Jones Detector: A traditional machine learning method using Haar-like features and a cascade classifier. While computationally light, it often lacks robustness in varying field conditions.
  2. MobileNetV3-SSDLite: A modern, lightweight Convolutional Neural Network (CNN) object detector. To fit the model within the GAP9's 128KB L1 memory, the authors implemented 8-bit quantization and a tiling strategy, using 2D DMA transfers to move data between L2 memory and the L1 scratchpad efficiently.
Architecture of the GAP9 SoC (A) and the evaluation board (B) used for pest detection.

The models were evaluated on a custom dataset of 158 images containing 1,275 labeled moths, split into "near" and "far" scenarios to test generalization across different camera distances.

Results and Discussion

The integration of the NE16 hardware accelerator proved to be a "great equalizer." Despite the inherent complexity of CNNs compared to hand-crafted feature methods, the MobileNetV3-SSDLite inference consumed nearly the same energy as the Viola-Jones algorithm (4.85 mJ vs 4.61 mJ per image).

Key performance metrics include:

  • Inference Speed: The CNN achieved a 147 ms inference time on the GAP9, representing a 9.5× speedup over the previous generation GAP8 MCU.
  • Accuracy: The CNN demonstrated superior generalization, maintaining 72% accuracy on the challenging "far" dataset, whereas Viola-Jones accuracy dropped to 45%.
  • Energy Efficiency: The system's low power consumption allows for an estimated battery life of 199 days on a 1000 mAh battery when processing an image every 30 seconds. This is up to 15× more efficient than previous state-of-the-art edge computing solutions like the Raspberry Pi.

Conclusion

The research demonstrates that modern heterogeneous MCUs like the GAP9 can bridge the gap between high-accuracy deep learning and ultra-low-power embedded systems. By leveraging dedicated hardware acceleration, complex CNNs can be deployed in the field for months on a single battery charge. Future work will focus on optimizing accelerator utilization for depthwise convolutions and improving software support for non-linear functions to further enhance performance.

Summary updated

Abstract

The codling moth pest poses a significant threat to global crop production, with potential losses of up to 80% in apple orchards. Special camera-based sensor nodes are deployed in the field to record and transmit images of trapped insects to monitor the presence of the pest. This paper investigates the embedding of computer vision algorithms in the sensor node using a novel State-of-the-Art Microcontroller Unit (MCU), the GreenWaves Technologies' GAP9 System-on-Chip, which combines 10 RISC-V general purposes cores with a convolution hardware accelerator. We compare the performance of a lightweight Viola-Jones detector algorithm with a Convolutional Neural Network (CNN), MobileNetV3-SSDLite, trained for the pest detection task. On two datasets that differentiate for the distance between the camera sensor and the pest targets, the CNN generalizes better than the other method and achieves a detection accuracy between 83% and 72%. Thanks to the GAP9's CNN accelerator, the CNN inference task takes only 147 ms to process a 320$\times$240 image. Compared to the GAP8 MCU, which only relies on general-purpose cores for processing, we achieved 9.5$\times$ faster inference speed. When running on a 1000 mAh battery at 3.7 V, the estimated lifetime is approximately 199 days, processing an image every 30 seconds. Our study demonstrates that the novel heterogeneous MCU can perform end-to-end CNN inference with an energy consumption of just 4.85 mJ, matching the efficiency of the simpler Viola-Jones algorithm and offering power consumption up to 15$\times$ lower than previous methods. Code at: https://github.com/Bomps4/TAFE_Pest_Detection
BibTeX
@article{bompani2024accelerating,
  author = {Luca Bompani and Luca Crupi and Daniele Palossi and Olmo Baldoni and Davide Brunelli and Francesco Conti and Manuele Rusci and Luca Benini},
  title = {{Accelerating Image-based Pest Detection on a Heterogeneous Multi-core Microcontroller}},
  journal = {IEEE Transactions on AgriFood Electronics},
  year = {2024},
  doi = {10.1109/tafe.2024.3451888},
}
Figure