Deep Metric Learning-Based Strawberry Disease Detection With Unknowns

Jie You, Kan Jiang, Joonwhoan LeeFrontiers in Plant Science 2022 · DOI

This research paper, published in Frontiers in Plant Science, presents a robust approach to a common headache in agricultural AI: what happens when a model encounters a disease it wasn't trained to recognize? By combining standard object detection with Deep Metric Learning (DML), the authors provide a framework for strawberry disease detection that handles "unknowns" with impressive reliability.

TL;DR

The researchers developed a two-stage pipeline that uses Deep Metric Learning to create a "feature space" where known strawberry diseases are tightly clustered, leaving room to identify previously unseen "unknown" diseases. This approach achieved a 93.7% mAP on real-world field data and provides a practical path for deploying plant pathology AI in unpredictable environments.

What Problem Does This Paper Address?

Most deep learning models for plant disease detection operate in a "closed-set" environment. They are trained on a specific list of diseases (e.g., powdery mildew, gray mold) and assume that everything they see must belong to one of those categories.

In the real world, this is a recipe for failure. A model might encounter a new mutation of a fungus, a rare nutrient deficiency, or simply a weirdly shaped leaf, and confidently misclassify it as a known disease. This "open-set" problem—where the model must distinguish between what it knows and what it doesn't—is the primary challenge the authors tackle.

Methods

The authors propose a two-stage "detect-then-filter" pipeline:

  1. Stage 1: Object Detection: They use a Faster R-CNN with a Feature Pyramidal Network (FPN). This stage is tuned with a low detection threshold to ensure high recall—essentially telling the model, "If it looks even remotely like a disease symptom, flag it."
  2. Stage 2: DML-Based Post-Filtering: The flagged regions (crops) are sent to a ResNet50 backbone trained using Deep Metric Learning (DML).
  3. The Loss Function: They use a combination of margin triplet loss and cross-entropy loss. Triplet loss forces the model to map similar disease symptoms close together in a high-dimensional space while pushing different diseases (and healthy plant parts) far apart.
  4. The Classifiers: The system uses two parallel classifiers in this second stage: a standard Softmax classifier (for known diseases) and a K-Nearest Neighbor (K-NN) classifier. The K-NN is the "secret sauce"—it includes "known unknown" samples in its database, allowing it to categorize a symptom as "unknown" if it doesn't closely match the clusters of known diseases.

Datasets

The study utilized a custom-built strawberry disease dataset:

  • Known Diseases: 8 categories including angular leafspot, anthracnose (fruit and runner), blossom blight, gray mold, leaf spot, and powdery mildew (fruit and leaf).
  • Normal Parts: 4 categories (healthy flower, fruit, leaf, and runner) used as "known unknowns" to help the model define the boundaries of disease.
  • Unknowns: A category of disease-like symptoms not covered by the 8 known classes.
  • Scale: 7,230 images total, with a specific field test set of 457 images collected from three different greenhouses to test real-world generalization.

Key Insights and Ideas

  • Creating "Empty Room": The core philosophy of using DML here isn't just better classification; it's about spatial management. By using triplet loss to make clusters of known diseases extremely compact, the researchers effectively leave "large empty rooms" in the feature space. When an unknown disease appears, its features fall into these empty spaces rather than being forced into a known cluster.
  • The Post-Filter Advantage: Standard object detectors often struggle with fine-grained classification (distinguishing between two similar-looking spots). By separating detection from classification, the authors can use a more powerful, specialized network (the DML-ResNet50) to scrutinize the regions of interest.
  • ROI Classifier Versatility: The authors point out that their second stage can function as a standalone "Region of Interest" (ROI) classifier. If a human scout manually crops a suspicious spot on a leaf, the DML/K-NN system can identify it without needing the first-stage detector.

Key Results

  • Field Performance: On real-world field data, the combined system achieved a 93.7% mAP for detecting known disease classes.
  • Accuracy: The DML-based K-NN classifier achieved 97.8% accuracy in distinguishing between known diseases, normal plant parts, and unknowns.
  • Recall for Unknowns: The system showed a high recall (91.7%) for unknown diseases, meaning it successfully flagged the vast majority of symptoms it hadn't been specifically trained on.
  • Performance Boost: Adding the DML post-filter improved the mean Average Precision (mAP) by approximately 2-3% compared to using a standard Faster R-CNN alone.

Limitations and Future Work

  • Bounding Box Dislocation: The authors noted that the first-stage detector sometimes produces slightly off-center bounding boxes. While they enlarged the boxes to provide more context to the second stage, extreme dislocation can still lead to classification errors.
  • Resolution Constraints: Very small symptoms, particularly on thin plant parts like runners, remain difficult to resolve and classify accurately.
  • Incremental Learning: While the model identifies unknowns, it doesn't currently "learn" them on the fly. The authors suggest that future work could involve incremental learning, where the model automatically incorporates these newly discovered "unknowns" into its "known" database over time.

Summary updated

Abstract

There has been substantial research that has achieved significant advancements in plant disease detection based on deep object detection models. However, with unknown diseases, it is difficult to find a practical solution for plant disease detection. This study proposes a simple but effective strawberry disease detection scheme with unknown diseases that can provide applicable performance in the real field. In the proposed scheme, the known strawberry diseases are detected with deep metric learning (DML)-based classifiers along with the unknown diseases that have certain symptoms. The pipeline of our proposed scheme consists of two stages: the first is object detection with known disease classes, while the second is a DML-based post-filtering stage. The second stage has two different types of classifiers: one is softmax classifiers that are only for known diseases and the <i>K</i>-nearest neighbor (<i>K</i>-NN) classifier for both known and unknown diseases. In the training of the first stage and the DML-based softmax classifier, we only use the known samples of the strawberry disease. Then, we include the known (<i>a priori</i>) and the known unknown training samples to construct the <i>K</i>-NN classifier. The final decisions regarding known diseases are made from the combined results of the two classifiers, while unknowns are detected from the <i>K</i>-NN classifier. The experimental results show that the DML-based post-filter is effective at improving the performance of known disease detection in terms of mAP. Furthermore, the separate DML-based <i>K</i>-NN classifier provides high recall and precision for known and unknown diseases and achieve 97.8% accuracy, meaning it could be exploited as a Region of Interest (ROI) classifier. For the real field data, the proposed scheme achieves a high mAP of 93.7% to detect known classes of strawberry disease, and it also achieves reasonable results for unknowns. This implies that the proposed scheme can be applied to identify disease-like symptoms caused by real known and unknown diseases or disorders for any kind of plant.
BibTeX
@inproceedings{you2022deep,
  author = {Jie You and Kan Jiang and Joonwhoan Lee},
  title = {{Deep Metric Learning-Based Strawberry Disease Detection With Unknowns}},
  booktitle = {Frontiers in Plant Science},
  year = {2022},
  doi = {10.3389/fpls.2022.891785},
}
Figure