Automatic and Accurate Calculation of Rice Seed Setting Rate Based on Image Segmentation and Deep Learning

Yixin Guo, Shuai Li, Zhanguo Zhang, Yang Li, Zhenbang Hu, Dawei Xin, Qingshan Chen, Jingguo Wang, Rongsheng ZhuFrontiers in Plant Science 2021 · DOI

TL;DR

Researchers have developed a high-throughput, deep learning-based system to automatically calculate the Rice Seed Setting Rate (RSSR)—a critical metric for predicting crop yield. By combining a YOLO V4 object detection model with a custom optimization algorithm to handle "partial" grains in cropped images, the system achieves an impressive 99.43% identification accuracy, turning a tedious manual task into a process that takes just seconds.

What Problem Does This Paper Address?

In rice breeding and agricultural management, the Rice Seed Setting Rate (RSSR) is a "make or break" phenotype. It is the ratio of full (fertile) grains to the total number of grains on a panicle. Traditionally, calculating this involves manual counting—a labor-intensive, error-prone process that creates a massive bottleneck for large-scale genetic analysis.

While computer vision offers a solution, rice panicles are complex. Grains are small, often overlap, and can be difficult to distinguish (full vs. empty) from 2D images. Previous attempts using 3D CT imaging were accurate but far too expensive and slow for field use. The challenge was to create a low-cost, non-invasive method using standard 2D photography that could match human accuracy.

Methods

The authors proposed a multi-stage pipeline to move from raw photos to a final RSSR percentage:

  1. Controlled Acquisition: Images were captured in a custom-built "cubed darkroom" using an iPhone X. This standardized lighting and background to reduce noise.
  2. Image Segmentation (Cropping): Because a single high-resolution image of a panicle contains hundreds of tiny grains, the authors cropped the original images into smaller patches. This prevents the "small object problem" where deep learning models lose detail on tiny features.
  3. Deep Learning Comparison: The study benchmarked six popular architectures: Faster R-CNN (with ResNet50 and VGG16 backbones), SSD, EfficientDet, YOLO V3, and YOLO V4.
  4. RSSROA Algorithm: The most novel contribution is the Rice Seed Setting Rate Optimization Algorithm. When you crop an image, some grains are inevitably cut in half. The RSSROA uses a weighted formula to account for these "half grains," ensuring they don't skew the final count. It uses a prior probability ($PH$) to estimate whether a partial grain is likely full or empty based on the distribution of whole grains in the sample.

Datasets

The researchers collected a diverse dataset to ensure the model could generalize:

  • Varieties: 56 different rice varieties.
  • Raw Data: 298 original high-resolution images (4,032 × 3,024 pixels) containing 263 rice panicles.
  • Processed Data: These were cropped into 4,560 smaller images.
  • Labels: Grains were manually labeled into three categories: Full, Empty, and Half-grain.

Key Insights and Ideas

  • The "Half-Grain" Solution: Most object detection papers ignore objects cut off by the image edge. In agricultural counting, this leads to significant undercounting. By treating "half-grains" as a distinct class and applying a statistical correction (RSSROA), the authors significantly closed the gap between automated and manual counts.
  • YOLO V4 Wins the Trade-off: While two-stage detectors like Faster R-CNN are often cited for accuracy, YOLO V4 proved superior here. It achieved the best balance of Mean Average Precision (mAP) and speed, likely due to its "bag of freebies" (data augmentation techniques like Mosaic) which helped it learn the subtle textures of empty vs. full grains.
  • Segmentation is Mandatory: The paper demonstrates that feeding a whole high-res image into a network results in dismal performance (mAP ~17%). Cropping the images to match the network's input size is the "aha" step that boosted accuracy to usable levels.

Key Results

  • Accuracy: The average identification accuracy reached 99.43%.
  • Correlation: The correlation between the automated RSSROA results and manual calculations was 93.21%.
  • Speed: The average processing time per image was approximately 2.65 seconds, significantly faster than manual counting.
  • Model Performance: YOLO V4 outperformed the other five models, particularly in detecting the difficult "empty grain" and "half grain" categories where other models (like SSD) failed almost entirely.

Limitations and Future Work

Despite the high accuracy, the authors noted a few hurdles:

  • Visual Mimicry: Some empty grains look remarkably like full grains due to their shape or the way they are shielded by other grains, leading to occasional misclassification.
  • Shielding/Overlap: While the 2D approach is fast, it cannot "see" grains completely hidden behind others.
  • Future Directions: The team plans to further refine the detection of full grains and explore ways to completely eliminate the impact of "half grains" on the final calculation. They also suggest this method could be adapted for other crops like wheat and soybeans.

Summary updated

Abstract

The rice seed setting rate (RSSR) is an important component in calculating rice yields and a key phenotype for its genetic analysis. Automatic calculations of RSSR through computer vision technology have great significance for rice yield predictions. The basic premise for calculating RSSR is having an accurate and high throughput identification of rice grains. In this study, we propose a method based on image segmentation and deep learning to automatically identify rice grains and calculate RSSR. By collecting information on the rice panicle, our proposed image automatic segmentation method can detect the full grain and empty grain, after which the RSSR can be calculated by our proposed rice seed setting rate optimization algorithm (RSSROA). Finally, the proposed method was used to predict the RSSR during which process, the average identification accuracy reached 99.43%. This method has therefore been proven as an effective, non-invasive method for high throughput identification and calculation of RSSR. It is also applicable to soybean yields, as well as wheat and other crops with similar characteristics.
BibTeX
@inproceedings{guo2021automatic,
  author = {Yixin Guo and Shuai Li and Zhanguo Zhang and Yang Li and Zhenbang Hu and Dawei Xin and Qingshan Chen and Jingguo Wang and Rongsheng Zhu},
  title = {{Automatic and Accurate Calculation of Rice Seed Setting Rate Based on Image Segmentation and Deep Learning}},
  booktitle = {Frontiers in Plant Science},
  year = {2021},
  doi = {10.3389/fpls.2021.770916},
}
Figure