TL;DR
This paper introduces a novel, efficient method for estimating Fusarium head blight (FHB) severity in wheat using non-invasive RGB images from consumer-grade cameras. They also release a new multiyear, multirater dataset, enabling models to generalize better across different years and rater biases, paving the way for accessible, automated disease phenotyping.
What Problem Does This Paper Address?
Fusarium head blight (FHB) is a major wheat disease causing significant yield losses and producing harmful mycotoxins. Accelerating resistance breeding for FHB is crucial, but current phenotyping methods are labor-intensive, time-consuming, and expensive. Traditional visual scoring relies on expert assessment, which is subjective and prone to interrater variability. While advanced techniques like hyperspectral imaging exist, they are often too costly and complex for widespread field use. The paper aims to bridge this gap by developing an efficient, non-invasive method using readily available RGB cameras that can automatically assess FHB severity, overcoming the limitations of current approaches.
Methods
The core of the proposed solution is a deep learning model based on a pre-trained EfficientNet-b0 architecture. This model was adapted into a regression task, directly predicting FHB severity scores from RGB images without requiring complex preprocessing like ear segmentation. The EfficientNet-b0 was chosen for its balance of accuracy and computational efficiency, making it suitable for potential deployment on mobile devices.
Key methodological aspects include:
- Regression Approach: Instead of classifying images into discrete FHB severity levels, the model predicts a continuous value, which is then rounded to assign a severity class. This approach, using L1 loss, was found to perform better than a classification task with categorical cross-entropy.
- Data Augmentation: To address the limited dataset size and class imbalance, standard data augmentation techniques like resizing, random rotation, random cropping, and horizontal flipping were employed. Notably, color space augmentation was omitted to preserve the visual cues of FHB.
- Weighted Random Sampling: To mitigate the impact of imbalanced class distribution (fewer samples for extreme FHB severities), a weighted random sampler was used during training to ensure that images from underrepresented classes were sampled more frequently.
- Transfer Learning: The EfficientNet-b0 model was pre-trained on the ImageNet dataset, and then fine-tuned on the FHB dataset. The initial layers were frozen for the first few epochs to adapt the network to the new output layer before unfreezing all weights for further training.
Datasets
The paper introduces a novel dataset comprising approximately 3,000 RGB images of wheat ears collected over three years (2020, 2021, and 2022). A key feature of this dataset is the multirater annotation, where each image was assessed by two independent raters. This allows for the evaluation of interrater reliability and the development of models that are robust to rater bias. The FHB severity is annotated on an ordinal scale from 1 to 9, which was later grouped into 6 classes ({≤2, 3, 4, 5, 6, ≥7}) for training due to limited samples at the extreme ends of the scale.
Key Insights and Ideas
- Interrater Reliability is a Bottleneck: The study highlights that interrater reliability (Cohen's kappa) is significantly lower than the achieved network-to-rater reliability. This suggests that human visual assessment, even by experts, is inherently subjective and less consistent than a well-trained model.
- Rater-Specific Models Outperform Averaging: Training separate models for each rater yielded better results than averaging their annotations. This implies that models can better adapt to the specific scoring biases of individual raters.
- Generalization Across Years is Crucial: Models trained on data from multiple years demonstrate improved performance when tested on unseen years, underscoring the importance of diverse training data for real-world applicability.
- RGB is Sufficient for FHB Severity: The research shows that consumer-grade RGB cameras, without complex preprocessing, can capture sufficient information for effective FHB severity estimation, making the technology more accessible.
- Regression Outperforms Classification for Ordinal Data: Treating FHB severity as an ordinal regression problem, rather than a strict classification task, led to better performance, likely because it accounts for the inherent order of the severity levels.
Key Results
- The developed EfficientNet-b0 regression model achieved competitive results, with network-to-rater reliability (Cohen's kappa) often exceeding interrater reliability. For instance, in 2020, network-to-rater kappa values reached 0.76, compared to interrater kappa of 0.68.
- Training on a combined dataset from 2020 and 2021 and testing on 2022 data significantly improved performance compared to single-year training. Specifically, the F1-weighted score increased by 0.14, accuracy by 0.11, kappa by 0.12, and RMSE decreased by 0.5.
- The model demonstrated good generalization capabilities, with performance metrics generally decreasing but remaining substantial when tested on data from years not included in training.
- The RMSE for the trained networks was often lower than the RMSE between raters, indicating that the models provided more consistent predictions than human assessments.
Limitations and Future Work
- Limited Data at Extremes: The dataset has fewer samples for very low and very high FHB severities, which can lead to overfitting and impact generalization. While grouping classes helped, more data at these extremes would be beneficial.
- Manual Image Cropping: The current workflow still requires manual cropping of images, which is time-consuming. Future work could explore automated plant segmentation to eliminate this step.
- Illumination Variability: While the model was trained under varying illumination conditions, further investigation into the impact of extreme lighting on predictive ability is suggested.
- Future Directions: The authors propose extending the models for mobile deployment, integrating them with UAVs for large-scale field screening, and exploring generative approaches to create synthetic data for further dataset augmentation. They also suggest utilizing the full 1-9 severity scale if more data at the extreme ends becomes available.
Summary updated