Automatic Quantification of Plant Disease From Field Image Data Using Deep Learning

Kanish Garg, Swati Bhugra, Brejesh LallWACV 2021

TL;DR

This paper introduces "Cascaded MRCNN," a novel deep learning framework for automatically quantifying plant disease severity directly from field images. By simultaneously segmenting individual leaf instances and their corresponding diseased regions, it offers a more precise and efficient alternative to manual inspection, crucial for accelerating plant breeding efforts.

What Problem Does This Paper Address?

Plant diseases are a major bottleneck in agricultural yield, and identifying disease-resistant plant varieties is key to mitigating these losses. Traditional methods for assessing disease severity rely on manual visual scoring by experts, which is time-consuming, prone to human error, and lacks scalability for large-scale breeding programs. While automated methods using imagery have emerged, they often focus on simply detecting the presence or absence of disease. However, plant breeders require more granular information: the precise location and extent of disease on individual leaves to accurately quantify severity. This is particularly challenging with field images, which are often plagued by complex backgrounds, uneven lighting, and overlapping plant structures, making robust segmentation difficult.

Methods

The core of this work is the Cascaded MRCNN architecture. This framework builds upon the Mask R-CNN architecture, which is adept at instance segmentation. The innovation lies in adding a "Cascaded Mask Branch" alongside the existing "Primary Mask Branch."

Here's how it works:

  1. Leaf Instance Segmentation: The "Primary Mask Branch" of the Cascaded MRCNN first identifies and segments individual leaf instances within the field image.
  2. Diseased Region Segmentation: Crucially, the "Cascaded Mask Branch" then leverages the feature maps generated during the leaf segmentation phase. Instead of processing the image anew, it reuses these features to semantically segment the diseased regions within each identified leaf instance. This "cascading" of feature maps allows for an end-to-end training process.
  3. Multi-Task Loss: The entire network is trained using a multi-task loss function that combines losses for classification, bounding box regression, the primary mask branch, and the cascaded mask branch. This encourages the network to learn both leaf and disease segmentation simultaneously.

This approach aims to overcome the limitations of previous methods that either required separate networks for leaf and disease segmentation (leading to redundant computations) or struggled with the complex real-world conditions of field imagery.

Datasets

The authors utilized a dataset of field images of maize plants infected with Northern Leaf Blight (NLB). This dataset was originally presented in [39]. The paper notes that the original dataset lacked precise disease region and leaf instance masks. Therefore, a significant contribution of this work was the generation of these detailed ground truth masks, which were manually labeled using the VGG Image Annotator (VIA) [10], guided by existing line annotations [9].

Key Insights and Ideas

  • End-to-End Disease Quantification: The most significant insight is the development of a single, unified framework that directly quantifies disease severity from raw field images, eliminating the need for separate, sequential processing steps.
  • Feature Map Re-utilization: The Cascaded MRCNN's clever use of feature maps from the leaf instance segmentation for diseased region segmentation is a key efficiency gain, reducing computational overhead and training complexity.
  • Addressing Real-World Challenges: The paper explicitly tackles the difficulties posed by complex backgrounds, uneven lighting, and leaf occlusion in field images, demonstrating the robustness of their approach in practical scenarios.
  • Granular Disease Information: By segmenting individual leaves and their diseased areas, the framework provides the precise, leaf-level severity data that plant breeders need, moving beyond simple presence/absence detection.

Key Results

The proposed Cascaded MRCNN framework demonstrated strong performance:

  • Disease Severity Correlation: Achieved a 73% correlation with manual ground truth data for disease severity index (SI).
  • Leaf Instance Segmentation: The primary mask branch of the Cascaded MRCNN achieved a Symmetric Best Dice (SBD) similarity score of 74% for leaf instance segmentation.
  • Diseased Region Segmentation: The cascaded mask branch achieved a Dice Accuracy of 91% for diseased region segmentation.
  • Runtime Efficiency: The framework achieved a run-time efficiency of 5 frames per second (5fps), indicating its potential for high-throughput applications.
  • Comparison: Outperformed baseline methods in both leaf instance segmentation (e.g., Mask R-CNN, Recurrent Attention) and diseased region segmentation (e.g., K-means, FCN, SegNet) in terms of accuracy metrics.

Limitations and Future Work

  • Generalizability: While the framework was tested on maize plants with Northern Leaf Blight, the authors suggest that the end-to-end architecture can be easily generalized for localizing and quantifying stress on other field plants.
  • Dataset Specificity: The current work is validated on a specific dataset. Future work could involve testing on a wider variety of crops, diseases, and environmental conditions to further prove its robustness.
  • Annotation Effort: The generation of detailed ground truth masks for leaf instances and diseased regions was a significant manual effort. Developing semi-automated or active learning approaches for annotation could further streamline the process for new datasets.

Summary updated

BibTeX
@inproceedings{garg2021automatic,
  author = {Kanish Garg and Swati Bhugra and Brejesh Lall},
  title = {{Automatic Quantification of Plant Disease From Field Image Data Using Deep Learning}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
}
Figure