AnoLeaf: Unsupervised Leaf Disease Segmentation via Structurally Robust Generative Inpainting

Swati Bhugra, Vinay Kaushik, Amit Gupta, Brejesh Lall, Santanu ChaudhuryWACV 2023

TL;DR

This paper introduces AnoLeaf, a novel unsupervised method for segmenting plant leaf diseases. By reframing disease segmentation as an anomaly localization task and employing a robust generative inpainting framework, AnoLeaf accurately identifies diseased regions without requiring pixel-level annotations, overcoming key limitations of existing supervised approaches.

What Problem Does This Paper Address?

Plant diseases pose a significant threat to agricultural productivity, necessitating efficient and accurate monitoring of plant health. Traditionally, this involves visual inspection of leaves, which is subjective and time-consuming. While deep learning has shown promise for automatic disease segmentation, supervised methods face several challenges: (1) heavily imbalanced datasets with far more healthy leaf images than diseased ones, (2) the labor-intensive and costly process of annotating disease regions at the pixel level, and (3) a lack of generalizability due to the diverse and heterogeneous nature of disease symptoms across different plant types and stages. This paper aims to address these limitations by developing an unsupervised approach that can accurately segment diseases without relying on extensive annotated data.

Methods

AnoLeaf is a two-stage framework that leverages generative inpainting for anomaly localization.

  1. Generative Inpainting Network: This module is trained exclusively on healthy leaf images to learn how to reconstruct them. It consists of two components:
  2. Coarse Network: A dilated convolutional network that produces an initial, somewhat blurry inpainted output of missing regions.
  3. Edge GAN: This component predicts a fine-grained edge map of the image, conditioned on the coarse inpainted output. By incorporating edge information, it helps to preserve structural details that might be lost in the coarse stage. This Edge GAN is trained adversarially with a feature-matching loss.
  4. Refinement Network: This network takes the combined information from the coarse output and the generated edge map to produce a more accurate and structurally robust inpainted image. It uses a contextual attention mechanism to focus on relevant spatial regions.
  5. Attention Map Module: After the inpainting network reconstructs a "healthy" version of the input image, the difference between the original diseased image and the inpainted image highlights potential anomaly (disease) regions. However, this difference map can be coarse. AnoLeaf enhances this by utilizing the contextual attention maps generated during the inpainting process. These maps highlight regions that the network did not need to attend to for inpainting, effectively indicating areas that deviate from the learned "normal" pattern. By integrating this attention map with the error map, AnoLeaf achieves more precise disease localization, even for subtle symptoms.

Datasets

The paper evaluates AnoLeaf on the following datasets:

  • Coffee dataset: A dataset of 372 top-down views of healthy and diseased coffee leaves.
  • MVTec Abnormal Detection dataset (MVTec AD): A benchmark dataset for industrial anomaly detection, containing images of 15 object and texture categories, with defect-free training images and test images with/without defects.
  • PlantVillage dataset: A large dataset of over 54,303 images of healthy and diseased leaves, categorized by plant species and diseases. This dataset was used for qualitative evaluation as it lacks pixel-level annotations.

Key Insights and Ideas

  • Reframing Disease Segmentation as Anomaly Localization: This is a crucial conceptual shift. By treating disease as an "anomaly" and training only on "normal" (healthy) data, the need for extensive disease-specific annotations is eliminated.
  • Structural Robustness through Edge Information: Incorporating edge prediction (Edge GAN) alongside image reconstruction significantly improves the structural integrity of the inpainted images, preventing the loss of fine details like leaf veins.
  • Contextual Attention for Precise Localization: The attention maps generated by the refinement network provide a powerful signal for pinpointing anomalies. By identifying regions the network didn't need to "fix," it effectively highlights what's different from the learned normal.
  • Generalizability Beyond Plant Diseases: The successful application of AnoLeaf on the MVTec AD dataset demonstrates its potential for general anomaly detection tasks, not just plant pathology.

Key Results

  • Quantitative Performance: AnoLeaf achieves state-of-the-art results on the Coffee dataset, outperforming traditional methods, weakly supervised methods, and other unsupervised anomaly localization techniques like AnoGAN and RIAD. For instance, on the Coffee dataset, AnoLeaf achieved a Precision of 0.9644, Recall of 0.8862, Jaccard Index of 0.7730, and F-value of 0.9236.
  • Qualitative Performance: Visual comparisons show that AnoLeaf produces more accurate and structurally consistent inpainted images compared to baselines, leading to better disease segmentation. It effectively captures fine details and avoids misclassifying leaf structures like veins as disease regions.
  • MVTec AD Results: AnoLeaf also demonstrated strong performance on the MVTec AD dataset, achieving a Precision of 0.93 and Recall of 0.92, surpassing other anomaly localization methods.

Limitations and Future Work

  • Computational Efficiency: While AnoLeaf is more efficient than iterative methods like AnoGAN, the paper notes that some baseline methods (like AnoGAN) rely on iterative backpropagation for latent space estimation, which can be computationally intensive. AnoLeaf's approach is presented as more efficient.
  • Generalizability to Diverse Plant Species: While AnoLeaf shows promise for generalizability, further extensive testing across a wider variety of plant species and disease types would be beneficial.
  • Subtle Symptom Detection: The paper suggests that improved segmentation could reveal subtle differences among plant species, implying that future work could focus on even finer-grained distinctions and early-stage disease detection.
  • Domain Knowledge: The unsupervised nature of AnoLeaf means it doesn't require domain-specific knowledge about disease symptoms, which is a significant advantage. Future work could explore how to best leverage this for even more robust and interpretable results.

Summary updated

BibTeX
@inproceedings{bhugra2023anoleaf,
  author = {Swati Bhugra and Vinay Kaushik and Amit Gupta and Brejesh Lall and Santanu Chaudhury},
  title = {{AnoLeaf: Unsupervised Leaf Disease Segmentation via Structurally Robust Generative Inpainting}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure