High-Throughput Rice Density Estimation from Transplantation to Tillering Stages Using Deep Networks

Liang Liu, Hao Lü, Yanan Li, Zhiguo CaoPlant 2020 · DOI

TL;DR

This paper introduces SFC²Net, a novel deep learning approach for high-throughput rice plant density estimation. By integrating a powerful backbone, multi-layer feature fusion, and a blockwise classification strategy, SFC²Net significantly improves accuracy and efficiency over manual counting and existing methods, even in challenging field conditions.

What Problem Does This Paper Address?

Accurately estimating rice plant density is crucial for yield prediction, crop health monitoring, and agricultural management. Traditionally, this involves tedious and error-prone manual counting. While computer vision offers automation potential, real-world agricultural environments present significant challenges: varying illumination, changes in plant appearance as they grow, and scale differences due to perspective. These factors make it difficult for existing computer vision methods to achieve reliable and efficient rice density estimation.

Methods

The core of this work is the Scale-Fusion Counting Classification Network (SFC²Net). It tackles the challenges through several key innovations:

  • MixNet-L Backbone: This leverages a pre-trained, multi-scale feature extraction network (MixNet-L) to capture rich visual information at different receptive fields, enhancing robustness to appearance variations.
  • Multilayer Feature Fusion (MFM): SFC²Net fuses feature maps from multiple layers of the backbone. This process combines high-level semantic information with low-level details, further improving feature representation and robustness.
  • Blockwise Classification for Sample Imbalance: To address the issue of scale variations causing sample imbalance (where some plant sizes are much more common than others), SFC²Net transforms the counting task into a blockwise classification problem. It quantizes counts into intervals, creating a "class map" that helps the model learn more effectively.
  • Redundant Processing and Deredundancy: A redundant module is used to generate a redundant class map, and then an inferring module performs inverse quantization and deredundancy to produce the final count map. This process helps to mitigate issues arising from object splitting and ensures a more accurate final count.

Datasets

The authors introduce a new Rice Plant Counting (RPC) dataset. This dataset consists of 382 high-resolution in-field rice images collected from two field sites in China between 2010 and 2013, covering the transplantation to tillering stages. The dataset features manual dot annotations at the root of each plant.

Key Insights and Ideas

  • Blockwise Classification is Key for Scale Invariance: The paper strongly suggests that transforming the counting problem into a classification of count intervals (blockwise classification) is a highly effective strategy for handling scale variations and sample imbalance, outperforming traditional regression-based methods.
  • Multi-Layer Fusion Enhances Robustness: Fusing features from multiple layers of a deep network is crucial for capturing both fine details and high-level semantics, leading to better performance in complex visual environments.
  • MixNet-L is a Powerful Backbone for Agricultural Phenotyping: The choice of MixNet-L as the backbone demonstrates its effectiveness in extracting discriminative features for plant counting tasks, outperforming more traditional backbones like VGG16.
  • High-Throughput is Achievable: The method is not just accurate but also fast, making it suitable for real-time or near-real-time applications in agricultural settings.

Key Results

SFC²Net achieved state-of-the-art performance on the RPC dataset, with:

  • Mean Absolute Error (MAE): 25.51
  • Mean Square Error (MSE): 38.06
  • Relative MAE (rMAE): 3.82%
  • R²: 0.98

This represents a significant improvement, with a 48.2% reduction in MAE compared to the conventional CSRNet approach. Furthermore, SFC²Net can process images at 16.7 frames per second for 1024x1024 images, highlighting its high-throughput capability. The method also demonstrated strong performance on the MTC dataset for maize tassel counting, indicating its generality.

Limitations and Future Work

The authors acknowledge several limitations:

  • Appearance Diversity: The current method shows relatively large errors on samples with significantly different plant appearances (e.g., tiny leaves, appearing as single points), suggesting a need for improved adaptation to other rice cultivars.
  • Quantization Errors: While blockwise classification is effective, quantization errors can still limit accuracy when the classification precision becomes very high.
  • Dataset Limitations: The RPC dataset primarily consists of images captured under non-rainy conditions and from a fixed camera perspective. This limits the model's adaptation to varying weather conditions, different growth stages beyond tillering, and diverse viewpoints (e.g., from drones or handheld devices).

Future work will focus on enriching the RPC dataset with more diverse conditions (weather, growth stages, perspectives, cultivars) and testing the method on other flexible platforms for real-world deployment.

Summary updated

Abstract

Rice density is closely related to yield estimation, growth diagnosis, cultivated area statistics, and management and damage evaluation. Currently, rice density estimation heavily relies on manual sampling and counting, which is inefficient and inaccurate. With the prevalence of digital imagery, computer vision (CV) technology emerges as a promising alternative to automate this task. However, challenges of an in-field environment, such as illumination, scale, and appearance variations, render gaps for deploying CV methods. To fill these gaps towards accurate rice density estimation, we propose a deep learning-based approach called the Scale-Fusion Counting Classification Network (SFC<sup>2</sup>Net) that integrates several state-of-the-art computer vision ideas. In particular, SFC<sup>2</sup>Net addresses appearance and illumination changes by employing a multicolumn pretrained network and multilayer feature fusion to enhance feature representation. To ameliorate sample imbalance engendered by scale, SFC<sup>2</sup>Net follows a recent blockwise classification idea. We validate SFC<sup>2</sup>Net on a new rice plant counting (RPC) dataset collected from two field sites in China from 2010 to 2013. Experimental results show that SFC<sup>2</sup>Net achieves highly accurate counting performance on the RPC dataset with a mean absolute error (MAE) of 25.51, a root mean square error (MSE) of 38.06, a relative MAE of 3.82%, and a <i>R</i> <sup>2</sup> of 0.98, which exhibits a relative improvement of 48.2% w.r.t. MAE over the conventional counting approach CSRNet. Further, SFC<sup>2</sup>Net provides high-throughput processing capability, with 16.7 frames per second on 1024 × 1024 images. Our results suggest that manual rice counting can be safely replaced by SFC<sup>2</sup>Net at early growth stages. Code and models are available online at https://git.io/sfc2net.
BibTeX
@inproceedings{liu2020high,
  author = {Liang Liu and Hao Lü and Yanan Li and Zhiguo Cao},
  title = {{High-Throughput Rice Density Estimation from Transplantation to Tillering Stages Using Deep Networks}},
  booktitle = {Plant},
  year = {2020},
  doi = {10.34133/2020/1375957},
}
Figure