Assessing Macro Disease Index of Wheat Stripe Rust Based on Segformer with Complex Background in the Field

Jie Deng, Xuan Lv, Lujia Yang, Baoqiang Zhao, Congying Zhou, Ziqian Yang, Jiarui Jiang, Ning Ning, Jinyu Zhang, Junzheng Shi, Zhanhong MaSensors 2022 · OpenReview

TL;DR

This paper introduces a deep learning approach using Segformer for accurate, field-level assessment of wheat stripe rust (WSR) disease severity. By addressing the challenge of imbalanced datasets and complex backgrounds, they developed a method to automatically calculate a "macro disease index" (MDI), offering a more objective and efficient alternative to traditional manual surveys.

What Problem Does This Paper Address?

Wheat stripe rust (WSR) is a significant agricultural threat, causing substantial yield losses worldwide. Accurately monitoring its severity, especially in the autumn growing stage, is crucial for predicting future outbreaks and implementing timely control measures. Traditional methods rely on manual visual estimation, which is subjective, time-consuming, and prone to errors, especially when dealing with large fields and complex backgrounds (like soil, weeds, or fallen leaves). Furthermore, the visual data itself presents a challenge: the diseased areas (rust) are often a small minority compared to healthy plant parts and background, creating a severe class imbalance problem for machine learning models. This paper aims to bridge this gap by developing an automated, accurate, and efficient method for WSR disease assessment at a canopy scale.

Methods

The core of this research lies in applying a deep learning semantic segmentation model, specifically Segformer, to identify and delineate WSR-infected areas on wheat leaves within complex field images. Segformer was chosen for its efficiency and state-of-the-art performance in semantic segmentation tasks.

To tackle the class imbalance issue, the researchers explored several strategies:

  • Data Augmentation: Generating synthetic variations of existing images to increase the representation of the minority rust class.
  • Under-sampling: Reducing the number of samples from the majority classes (healthy and other) to balance the dataset.
  • Loss Functions: Experimenting with various loss functions (e.g., cross-entropy, dice loss, focal loss) known to perform better with imbalanced data.
  • Class Weighting: Adjusting the importance of different classes during model training.

The segmented disease areas are then used to automatically calculate a Macro Disease Index (MDI). This index quantifies the overall disease severity across a larger area, moving beyond single-leaf analysis to a more practical canopy-level assessment.

Datasets

The study utilized a custom dataset of winter wheat images collected in Gangu County, Tianshui City, Gansu Province, China, during late November 2020. These images were captured at a canopy scale under complex field backgrounds, including mulch films, soil, snow, weeds, dried leaves, and overlapping foliage.

  • Original Dataset (O): Consisted of 25,530 images of size 256x256 pixels, manually annotated to distinguish between healthy, rust, and other (background) classes. This dataset exhibited a severe class imbalance, with the rust class representing only 5.66% of the pixels.
  • Augmented Dataset (A): Created by applying data augmentation techniques to the original training and validation sets, resulting in 43,530 pairs. This significantly increased the rust class proportion to 17.89%.
  • Under-sampled Datasets (U1, U2, U3): Three datasets were created by randomly sampling images from the original dataset to achieve different degrees of class balancing.

Key Insights and Ideas

  • Segformer's Superiority: The Segformer architecture emerged as the best-performing model for WSR segmentation among several tested deep learning frameworks, achieving a rust class F1 score of 72.60% on the original imbalanced dataset.
  • Loss Functions Fall Short: Standard techniques like re-weighting minority classes or using specialized loss functions (e.g., focal loss) alone were found to be largely ineffective in significantly improving the model's ability to identify the rust class in the face of severe class imbalance.
  • Re-sampling is Key: The most impactful strategy for alleviating class imbalance was re-sampling, specifically through data augmentation of the minority class or under-sampling. This approach dramatically improved the rust class F1 score to 86.6%.
  • MDI as a Robust Metric: The Macro Disease Index (MDI) proved to be a reliable quantitative indicator for assessing disease severity at the canopy level, correlating strongly with manual assessments (R² = 0.992).
  • Complex Backgrounds are Manageable: The study successfully demonstrated that deep learning models, particularly Segformer, can effectively segment diseased areas even in challenging field conditions with diverse backgrounds.

Key Results

  • Segmentation Performance: Segformer achieved the highest rust class F1 score of 72.60% on the original dataset. After applying data augmentation, the rust class F1 score improved to 86.6%.
  • MDI Accuracy: The model trained on the augmented dataset achieved the best performance in predicting the Macro Disease Index, with a coefficient of determination (R²) of 0.992 and a Root Mean Squared Error (RMSE) of 0.008 when compared to reference data.
  • Class Imbalance Impact: The study highlighted that the imbalanced nature of the data significantly hindered the accurate identification of the rust class.

Limitations and Future Work

  • Temporal Limitations: The study focused on images taken during the autumn tillering stage. The model's applicability to wheat at later growth stages (e.g., jointing stage) where leaves might be less flat and more overlapping needs further investigation.
  • Disease Specificity: The current dataset and model are specific to wheat stripe rust. They do not account for other wheat diseases or insect pests that might present similar visual symptoms, which could lead to misclassification.
  • Future Work: The authors envision developing a mobile application that allows users to capture field images and automatically obtain WSR disease index estimations. This would provide valuable data for a national WSR prediction and forecasting database. They also suggest that future work could explore generative adversarial networks (GANs) for generating minority samples to further address data imbalance.

Summary updated

Abstract

Wheat stripe rust (WSR) is a foliar disease that causes destructive damage in the wheat production context. Accurately estimating the severity of WSR in the autumn growing stage can help to objectively monitor the disease incidence level of WSR and predict the nationwide disease incidence in the following year, which have great significance for controlling its nationwide spread and ensuring the safety of grain production. In this study, to address the low accuracy and the efficiency of disease index estimation by traditional methods, WSR-diseased areas are segmented based on Segformer, and the macro disease index (MDI) is automatically calculated for the measurement of canopy-scale disease incidence. The results obtained with different semantic segmentation algorithms, loss functions, and data sets are compared for the segmentation effect, in order to address the severe class imbalance in disease region segmentation. We find that: (1) The results of the various models differed significantly, with Segformer being the best algorithm for WSR segmentation (rust class F1 score = 72.60%), based on the original data set; (2) the imbalanced nature of the data has a significant impact on the identification of the minority class (i.e., the rust class), for which solutions based on loss functions and re-weighting of the minority class are ineffective; (3) data augmentation of the minority class or under-sampling of the original data set to increase the proportion of the rust class greatly improved the F1-score of the model (rust class F1 score = 86.6%), revealing that re-sampling is a simple and effective approach to alleviating the class imbalance problem. Finally, the MDI was used to evaluate the models based on the different data sets, where the model based on the augmented data set presented the best performance (R2 = 0.992, RMSE = 0.008). In conclusion, the deep-learning-based semantic segmentation method, and the corresponding optimization measures, applied in this study allow us to achieve pixel-level accurate segmentation of WSR regions on wheat leaves, thus enabling accurate assessment of the degree of WSR disease under complex backgrounds in the field, consequently providing technical support for field surveys and calculation of the disease level.
BibTeX
@inproceedings{deng2022assessing,
  author = {Jie Deng and Xuan Lv and Lujia Yang and Baoqiang Zhao and Congying Zhou and Ziqian Yang and Jiarui Jiang and Ning Ning and Jinyu Zhang and Junzheng Shi and Zhanhong Ma},
  title = {{Assessing Macro Disease Index of Wheat Stripe Rust Based on Segformer with Complex Background in the Field}},
  booktitle = {Sensors},
  year = {2022},
  url = {https://openreview.net/forum?id=W4rdhrUmW8},
}
Figure