Wheat Ear Recognition Based on RetinaNet and Transfer Learning

Jingbo Li, Changchun Li, Shuaipeng Fei, Chunyan Ma, Weinan Chen, Fan Ding, Yilin Wang, Yacong Li, Jinjin Shi, Zhen Xiao2021 · DOI

TL;DR

This paper tackles the challenge of accurately counting wheat ears for yield estimation, a task crucial for global food security. By leveraging deep learning models like RetinaNet and Faster R-CNN, combined with transfer learning, the authors achieve high accuracy and efficiency in wheat ear recognition, outperforming previous methods and demonstrating robustness across different growth stages.

What Problem Does This Paper Address?

Accurate estimation of wheat yield is vital for global food security, and the number of wheat ears is a key indicator for this. Traditional methods for counting wheat ears are labor-intensive, time-consuming, and often inaccurate, especially in diverse field conditions. Furthermore, the visual characteristics of wheat ears change significantly throughout their growth stages, making consistent identification challenging. Existing deep learning approaches have shown promise but often struggle with real-world variability, require extensive training data, and may not be suitable for real-time applications like those needed for unmanned aerial vehicles (UAVs). This paper aims to develop a robust wheat ear recognition model that is accurate across different growth stages, efficient, and adaptable to varying field conditions.

Methods

The study investigates two prominent deep learning object detection architectures: Faster R-CNN and RetinaNet.

  • Faster R-CNN is a two-stage detector known for its accuracy, employing a Region Proposal Network (RPN) to identify potential object locations before classification.
  • RetinaNet is a one-stage detector that combines the strengths of anchor-based methods and feature pyramids (FPN) to achieve high accuracy and speed, particularly for small objects. It addresses the class imbalance issue in dense object detection using a novel "focal loss" function.

A key aspect of this research is the application of transfer learning. Pre-trained models on a large dataset (Global WHEAT) are used as a starting point, and then fine-tuned on specific wheat ear datasets. This approach helps overcome the challenge of limited labeled data in real-world agricultural settings. Data augmentation techniques, including image denoising, enhancement, flipping, and color space transformations, were also employed to improve model robustness and generalization.

Datasets

The study utilizes two primary datasets:

  1. Global WHEAT dataset: This is a large-scale dataset containing 3422 images of wheat ears from various growth environments and varieties, collected by experts over a year. It serves as the primary training and pre-training dataset.
  2. Custom collected dataset: This dataset consists of 715 high-definition digital images of wheat ears captured in the field during the filling and mature stages. This dataset is used for testing the models' performance in more realistic, field-specific conditions.

Key Insights and Ideas

  • RetinaNet's Superiority: RetinaNet consistently outperforms Faster R-CNN in terms of both recognition accuracy and speed, especially when dealing with the complexities of wheat ear identification across different growth stages.
  • Transfer Learning is Crucial: Applying transfer learning significantly boosts the performance of both models, demonstrating its effectiveness in overcoming data scarcity in agricultural applications.
  • Growth Stage Matters: Wheat ear characteristics change significantly with growth, impacting recognition accuracy. Models trained on specific stages may not generalize well to others without careful consideration.
  • Data Augmentation Enhances Robustness: Techniques like denoising and image enhancement are vital for improving model performance and making it more resilient to variations in lighting and environmental conditions.
  • Small Object Detection Advantage: RetinaNet's architecture, particularly its use of FPN, provides an advantage in detecting small objects like wheat ears, contributing to its higher accuracy.

Key Results

  • Global WHEAT Dataset Performance: RetinaNet achieved an average accuracy of 0.82, while Faster R-CNN achieved 0.72 on the Global WHEAT dataset.
  • Custom Dataset Performance (Transfer Learning): After transfer learning, RetinaNet achieved an R² of 0.9722, and Faster R-CNN achieved 0.8702 on the custom collected dataset.
  • Filling Stage Accuracy: RetinaNet achieved an R² of 0.907 in the filling stage, significantly higher than Faster R-CNN's 0.792.
  • Mature Stage Accuracy: RetinaNet achieved an R² of 0.514 in the mature stage, while Faster R-CNN achieved 0.844. This highlights the challenge of generalization across stages.
  • Recognition Speed: RetinaNet demonstrated a faster average recognition time of 6.51 seconds compared to Faster R-CNN's 9.19 seconds for processing 180 images.
  • F1-Score: RetinaNet achieved a higher F1-score (91.17%) compared to Faster R-CNN (82.25%).

Limitations and Future Work

  • Generalization Across Stages: While RetinaNet shows promise, its performance can degrade when identifying wheat ears in stages different from those it was primarily trained on. The study notes that the causes for performance differences between models across growth stages are not fully understood and warrant further exploration.
  • Model Optimization: The paper suggests that optimizing the feature extraction network (ResNet50) used in RetinaNet could further improve wheat recognition accuracy.
  • UAV Integration: The authors aim to investigate wheat ear recognition using images captured by unmanned aerial vehicles (UAVs) in future work, which would be a significant step towards real-time yield estimation and precision agriculture.
  • Small Sample Size Studies: The study highlights the need for further research into how training sample size and selection methods impact model performance, especially for scenarios with limited data.

Summary updated

BibTeX
@misc{li2021wheat,
  author = {Jingbo Li and Changchun Li and Shuaipeng Fei and Chunyan Ma and Weinan Chen and Fan Ding and Yilin Wang and Yacong Li and Jinjin Shi and Zhen Xiao},
  title = {{Wheat Ear Recognition Based on RetinaNet and Transfer Learning}},
  year = {2021},
  doi = {10.3390/s21144845},
}
Figure