Computer Vision with Deep Learning for Plant Phenotyping in Agriculture: A Survey

Akshay L. Chandra, Sai Vikas Desai, Wei Guo, Vineeth N. BalasubramanianarXiv 2020 · DOI · OpenReview

TL;DR

This survey provides a comprehensive overview of how deep learning is revolutionizing plant phenotyping in agriculture. By leveraging advanced computer vision techniques, researchers are developing more efficient and accurate methods for monitoring crops, ultimately aiming to boost agricultural productivity and address global food demand challenges.

What Problem Does This Paper Address?

The world faces a growing food demand, exacerbated by declining arable land and the need for increased agricultural efficiency. Precision agriculture offers a solution by enabling data-driven crop management, but accurate monitoring of plant traits (phenotyping) is crucial for this. Traditional phenotyping methods are slow, labor-intensive, and inefficient. While machine learning has improved this, it often relies on handcrafted features and struggles in uncontrolled environments. Deep learning, with its ability to automatically extract features from raw data, presents a powerful opportunity to overcome these limitations and advance plant phenotyping for better crop management and yield.

Methods

This paper surveys the application of deep learning (DL) in plant phenotyping. It contrasts traditional machine learning (ML) approaches, which involve manual feature extraction and classification/regression, with DL methods that automate feature learning. The survey highlights the role of Convolutional Neural Networks (CNNs) as a key DL architecture for vision-based tasks like image classification, object detection, and semantic segmentation, which are fundamental to plant phenotyping. It also touches upon various DL approaches such as supervised, semi-supervised, and unsupervised learning, and discusses their relevance to plant phenotyping tasks.

Datasets

The paper references several datasets used in plant phenotyping research, including:

  • CropDeep [42]: Used for detection tasks.
  • Rosette Plant Phenotyping dataset [43, 44]: Provides multi-modal annotations for detection, segmentation, and leaf center.
  • Arabidopsis datasets [43-45]: Used for various phenotyping tasks.
  • Sorghum-Head [37] and Wheat-Panicle [46]: Datasets for specific crop-related tasks.
  • Crop/Weed segmentation [24] and Crop/Tassle segmentation [47]: Datasets for segmentation tasks.
  • ImageNet [137]: A general-purpose dataset used for pre-training models in transfer learning.
  • LifeCLEF [138] 2015 challenge dataset: Used for plant phenotyping tasks.

The paper also discusses the general challenge of acquiring large, labeled datasets for specific agricultural conditions, which often necessitates researchers creating their own datasets.

Key Insights and Ideas

  • Deep Learning's Automation Power: Deep learning significantly simplifies plant phenotyping by automating feature extraction, eliminating the need for manual, domain-expert-driven feature engineering that was a bottleneck in traditional ML.
  • CNNs as the Backbone: Convolutional Neural Networks (CNNs) are exceptionally well-suited for visual plant phenotyping due to their ability to effectively process spatial information in images, making them ideal for tasks like disease detection, fruit counting, and yield estimation.
  • Bridging the Gap to Precision Agriculture: By enabling high-throughput and accurate plant monitoring, deep learning directly supports the goals of precision agriculture, allowing for more informed and customized crop management decisions.
  • Addressing Data Scarcity: The survey highlights ongoing research into techniques like data augmentation, weakly supervised learning, and transfer learning to mitigate the challenge of limited labeled data in agricultural datasets.
  • The Importance of Explainability: A significant open problem is the "black box" nature of deep neural networks. Developing explainable AI models for plant phenotyping is crucial for building trust and understanding the underlying reasons for plant traits, which can lead to breakthroughs in plant behavior research.

Key Results

The paper is a survey, so it doesn't present new experimental results. Instead, it synthesizes findings from numerous research papers. The key "results" are the demonstrated successes of deep learning in various plant phenotyping applications, including:

  • Improved accuracy and efficiency: Deep learning models have achieved state-of-the-art performance in tasks like weed detection, fruit/organ counting, disease detection, and yield estimation.
  • Enabling new capabilities: DL has made previously difficult phenotyping tasks possible, such as automated crop monitoring from UAVs and satellites, and detailed analysis of plant traits.
  • Advancements in specific tasks: The survey points to research showing promising results in areas like panicle/spike detection, leaf counting, fruit detection, and pixel-wise segmentation.

Limitations and Future Work

The authors identify several limitations and areas for future research:

  • Training Data Bottleneck: The reliance of deep learning on large, labeled datasets remains a significant challenge. Future work should focus on developing more data-efficient models and adapting techniques like unsupervised, self-supervised, weakly supervised, active, and semi-supervised learning.
  • Explainability: Deep neural networks are often considered "black boxes." Developing explainable models is crucial for debugging, understanding predictions, and gaining deeper insights into plant behavior.
  • Data Collection Challenges: Vision-based phenotyping faces issues with occlusion, 3D reconstruction inaccuracies, and varying lighting conditions. Future tools need to be robust to these visual variations.
  • Field Environment Variations: The large variations in field environments are difficult to fully capture, limiting the generalizability of models.
  • Cost and Maintenance: High-throughput phenotyping tools can be prohibitively expensive for widespread adoption, especially in developing countries.

The paper concludes by emphasizing that it's an exciting time for plant phenotyping research, with advances in deep learning offering rapid progress.

Summary updated

Abstract

In light of growing challenges in agriculture with ever growing food demand across the world, efficient crop management techniques are necessary to increase crop yield. Precision agriculture techniques allow the stakeholders to make effective and customized crop management decisions based on data gathered from monitoring crop environments. Plant phenotyping techniques play a major role in accurate crop monitoring. Advancements in deep learning have made previously difficult phenotyping tasks possible. This survey aims to introduce the reader to the state of the art research in deep plant phenotyping.
BibTeX
@misc{chandra2020computer,
  author = {Akshay L. Chandra and Sai Vikas Desai and Wei Guo and Vineeth N. Balasubramanian},
  title = {{Computer Vision with Deep Learning for Plant Phenotyping in Agriculture: A Survey}},
  year = {2020},
  eprint = {2006.11391},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.11391},
}
Figure