New Method for Tomato Disease Detection Based on Image Segmentation and Cycle-GAN Enhancement

Anjun Yu, Yonghua Xiong, Zirong Lv, Peng Wang, Jinhua She, Longsheng WeiSensors 2024 · DOI

TL;DR

This paper tackles the challenge of limited and noisy datasets in agricultural AI by introducing a novel method for tomato disease detection. It combines an automatic leaf segmentation technique (AISG) to isolate disease-relevant features from background noise with Cycle-GAN for cross-category image enhancement, achieving a remarkable 98.61% accuracy on ten tomato disease types.

What Problem Does This Paper Address?

Developing accurate deep learning models for plant disease detection is heavily reliant on large, high-quality datasets. However, acquiring such datasets in agriculture is challenging due to the cost, labor, and the inherent variability of real-world environments. Existing methods often struggle with:

  • Limited Datasets: Insufficient data leads to poor model generalization and accuracy.
  • Noise Interference: Images captured in natural settings are prone to background clutter (e.g., hands, soil, other leaves) that can obscure disease symptoms and confuse models.
  • Data Augmentation Limitations: Traditional augmentation techniques (like rotation or brightness changes) often produce highly correlated samples, limiting the generation of truly novel features and potentially leading to overfitting.
  • Cross-Category Enhancement Issues: While GANs can generate synthetic data, existing methods often enhance within the same category or struggle to isolate specific regions, potentially introducing unwanted noise into the generated images.

This paper aims to overcome these limitations to improve the accuracy and robustness of tomato disease detection systems.

Methods

The proposed method integrates three key components:

  1. Automatic Leaf Segmentation Algorithm (AISG): This algorithm, built upon the EISeg interactive segmentation method, is designed to automatically isolate tomato leaves from complex backgrounds. It incorporates pre-processing steps, including a "super-green factor" calculation, to better distinguish foreground (leaf) from background. This step is crucial for removing environmental noise and focusing the model's attention on disease-relevant leaf features.
  2. Cycle-GAN for Cross-Category Image Enhancement: Cycle-GAN is employed to address the issue of limited samples by performing cross-category image transformation. This allows the model to generate new, diverse training samples that mimic disease characteristics, even when starting from healthy leaf images or images with subtle symptoms. The segmentation step preceding Cycle-GAN ensures that the transformation is applied only to the target leaf area, preventing unwanted background changes.
  3. MobileNetV3 for Classification: A pre-trained MobileNetV3 model is used for the final disease classification. MobileNetV3 is chosen for its lightweight and efficient architecture, making it suitable for deployment on mobile devices, which aligns with the goal of enabling farmers to diagnose diseases in the field. Transfer learning is utilized to leverage the model's existing knowledge from ImageNet.

Datasets

  • PlantVillage Dataset: This publicly available dataset, containing over 54,000 images of 14 plant types and 26 diseases, served as the initial source.
  • Newly Constructed Dataset: The authors augmented the PlantVillage dataset by:
  • Selecting 8000 laboratory images.
  • Collecting 1824 field images from various search engines.
  • Acquiring 1892 images from local planting bases in complex environments.
  • Applying traditional augmentation techniques (horizontal, vertical, random, and reverse folding) to generate an additional 3502 images.
  • Generating 3100 synthetic images using Cycle-GAN.

The final augmented dataset comprises 18,318 images, divided into training, validation, and testing sets in an 8:1:1 ratio.

Key Insights and Ideas

  • Segmentation as a Precursor to Enhancement: The paper highlights that performing image segmentation before applying generative enhancement (like Cycle-GAN) is critical. This prevents the GAN from introducing unwanted noise or transforming irrelevant background elements, thereby improving the quality and relevance of the generated synthetic data.
  • "Super-Green Factor" for Robust Segmentation: The introduction of a "super-green factor" (2G-R-B) as a pre-processing step for segmentation demonstrates a clever way to leverage inherent color characteristics of leaves to better differentiate them from backgrounds, especially in challenging real-world scenarios.
  • Cycle-GAN for Targeted Enhancement: By combining Cycle-GAN with segmentation, the authors achieve a more controlled and effective data augmentation strategy. This allows for generating diverse disease-like features without being limited to within-category transformations, which is a significant step beyond traditional augmentation.
  • MobileNetV3 for Practical Deployment: The choice of MobileNetV3 underscores the practical applicability of the research, aiming for a solution that can be readily used by farmers on their mobile devices.

Key Results

  • Classification Accuracy: The proposed method achieved an impressive classification accuracy of 98.61% for ten types of tomato diseases on the augmented dataset. This significantly outperforms the accuracy on the original PlantVillage dataset (95.74%).
  • AISG Performance: The AISG algorithm demonstrated a 7% improvement in accuracy and reduced segmentation time by approximately 3 seconds compared to the EISeg algorithm.
  • Cycle-GAN Quality: Quantitative metrics like PSNR and MS-SSIM indicated that the images generated by Cycle-GAN closely resembled the quality and structure of the original images, suggesting effective and realistic synthetic data generation.
  • Comparative Performance: The proposed method (DMD) outperformed other models (DMA, DMB, DMC) and existing approaches in terms of accuracy, precision, recall, and F1-score. It also achieved the shortest training time among compared methods.
  • Generalization: The model showed strong generalization ability, achieving high accuracy (above 97%) on various other plant disease datasets (grape, tobacco, maize, apple) and even on the ImageNetDogs dataset.

Limitations and Future Work

  • Multi-Disease Identification: The paper acknowledges that the current method does not explicitly handle images with two or more diseases appearing simultaneously on a single leaf. This is identified as a limitation for practical applications.
  • Future Research Directions:
    • Further optimization of the model to accurately identify multi-disease leaves.
  • Integration of the disease recognition and classification method into mobile terminals to create a practical tool for smart agriculture.

Summary updated

Abstract

A major concern in data-driven deep learning (DL) is how to maximize the capability of a model for limited datasets. The lack of high-performance datasets limits intelligent agriculture development. Recent studies have shown that image enhancement techniques can alleviate the limitations of datasets on model performance. Existing image enhancement algorithms mainly perform in the same category and generate highly correlated samples. Directly using authentic images to expand the dataset, the environmental noise in the image will seriously affect the model's accuracy. Hence, this paper designs an automatic leaf segmentation algorithm (AISG) based on the EISeg segmentation method, separating the leaf information with disease spot characteristics from the background noise in the picture. This algorithm enhances the network model's ability to extract disease features. In addition, the Cycle-GAN network is used for minor sample data enhancement to realize cross-category image transformation. Then, MobileNet was trained by transfer learning on an enhanced dataset. The experimental results reveal that the proposed method achieves a classification accuracy of 98.61% for the ten types of tomato diseases, surpassing the performance of other existing methods. Our method is beneficial in solving the problems of low accuracy and insufficient training data in tomato disease detection. This method can also provide a reference for the detection of other types of plant diseases.
BibTeX
@inproceedings{yu2024new,
  author = {Anjun Yu and Yonghua Xiong and Zirong Lv and Peng Wang and Jinhua She and Longsheng Wei},
  title = {{New Method for Tomato Disease Detection Based on Image Segmentation and Cycle-GAN Enhancement}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24206692},
}
Figure