Early and Accurate Detection of Tomato Leaf Diseases Using TomFormer

Asim Khan, Umair Nawaz, Kshetrimayum Lochan, Lakmal D. Seneviratne, Irfan Hussain

TL;DR

Researchers have developed TomFormer, a hybrid AI model that combines the local feature-extraction power of Convolutional Neural Networks (CNNs) with the global "big picture" perspective of Transformers to detect tomato leaf diseases. Deployed on a mobile Hello Stretch robot, the system achieved state-of-the-art accuracy (up to 87% mAP), offering a path toward autonomous, real-time disease monitoring in greenhouses.

What Problem Does This Paper Address?

Tomato crops are a global staple, but they are highly susceptible to pathogens that can devastate yields. For farmers, early detection is the difference between a minor intervention and a total loss.

While deep learning has revolutionized plant pathology, two main hurdles remain:

  1. Model Limitations: Traditional CNNs are great at identifying local textures (like a specific spot) but often struggle to understand the spatial relationship between symptoms across a whole leaf.
  2. Deployment Gap: Most models live in labs. There is a lack of integrated systems that can actually move through a greenhouse, capture high-quality data, and provide immediate feedback to growers.

Methods

The authors introduce TomFormer, a "fusion" architecture designed to get the best of both worlds in computer vision:

  • The Hybrid Backbone: The model uses a CNN block to extract low-level features (textures and edges) and a Transformer encoder to capture "long-range dependencies." This allows the AI to see both the tiny lesions and how they are distributed across the entire leaf structure.
  • Object Queries: Borrowing from the DEtection TRansformer (DETR) paradigm, TomFormer uses 20 "learnable object queries." Instead of scanning an image with thousands of overlapping boxes (like older methods), the model asks 20 specific "questions" about the image to locate and identify up to 15 leaves simultaneously.
  • Robotic Integration: The model was integrated into the Hello Stretch robot—a slender, mobile manipulator equipped with an Intel RealSense RGB-D camera and a Jetson Nano computer. This allows the system to navigate indoor environments and capture high-resolution images of leaves from various angles.

Datasets

The study evaluated TomFormer on three distinct datasets:

  1. KUTomaDATA: A new, proprietary dataset of 939 images collected via mobile phones in a greenhouse in Abu Dhabi, UAE. It covers eight classes, including healthy leaves and diseases like Late Blight and Mosaic Virus.
  2. PlantDoc: A public dataset containing 2,598 data points across 13 plant species.
  3. PlantVillage: A well-known benchmark featuring over 50,000 images captured under controlled, laboratory conditions.

Key Insights and Ideas

  • The Power of Fusion: The "aha moment" here is that neither Transformers nor CNNs are perfect for agriculture on their own. By concatenating CNN-derived features with Transformer patch embeddings, the model maintains high spatial resolution while understanding global leaf patterns.
  • Set Prediction vs. Bounding Boxes: By treating detection as a "direct set prediction" problem (using the Hungarian algorithm for bipartite matching loss), the model avoids the complex post-processing steps (like Non-Maximum Suppression) required by models like YOLO.
  • Context Matters: The model performed best on the PlantVillage dataset because of its uniform backgrounds, but the authors emphasize that the KUTomaDATA results (87% mAP) are more significant because they reflect "messy" real-world greenhouse conditions.

Key Results

TomFormer consistently outperformed established models like YOLOS, DETR, ViT, and Swin Transformer across all benchmarks:

  • Accuracy: Achieved a Mean Average Precision (mAP) of 87% on KUTomaDATA, 81% on PlantDoc, and 83% on PlantVillage.
  • Efficiency: The model is relatively lightweight with only 1.30 million parameters, making it suitable for edge deployment on robotic platforms.
  • Class Performance: It was particularly effective at identifying "Healthy leaves" (92% mAP) and "Bacterial spots" (90% mAP), though it found "Early blight" (80% mAP) more challenging due to its visual similarity to other diseases.

Limitations and Future Work

  • Inference Speed: The paper notes an estimated inference time of approximately 200 seconds. While the authors describe the system as "real-time," this latency is quite high for a mobile robot and likely refers to the total processing pipeline or contains a typographical error (as 1.3M parameters usually run much faster).
  • Dataset Size: While the results are promising, the custom KUTomaDATA dataset is relatively small (under 1,000 images). Expanding this to include more growth stages and lighting conditions would improve robustness.
  • Future Directions: The authors suggest that further fine-tuning and adapting the model to new, larger datasets will be key to scaling this technology for commercial tomato production.

Summary updated

Abstract

Tomato leaf diseases pose a significant challenge for tomato farmers, resulting in substantial reductions in crop productivity. The timely and precise identification of tomato leaf diseases is crucial for successfully implementing disease management strategies. This paper introduces a transformer-based model called TomFormer for the purpose of tomato leaf disease detection. The paper's primary contributions include the following: Firstly, we present a novel approach for detecting tomato leaf diseases by employing a fusion model that combines a visual transformer and a convolutional neural network. Secondly, we aim to apply our proposed methodology to the Hello Stretch robot to achieve realtime diagnosis of tomato leaf diseases. Thirdly, we assessed our method by comparing it to models like YOLOS, DETR, ViT, and Swin, demonstrating its ability to achieve state-of-the-art outcomes. For the purpose of the experiment, we used three datasets of tomato leaf diseases, namely KUTomaDATA, PlantDoc, and Plan Village, where KUTomaDATA is being collected from a greenhouse in Abu Dhabi, UAE. Finally, we present a comprehensive analysis of the performance of our model and thoroughly discuss the limitations inherent in our approach. TomFormer performed well on the KUTomaDATA, PlantDoc, and PlantVillage datasets, with mean average accuracy (mAP) scores of 87%, 81 %, and 83 %, respectively. The comparative results in terms of mAP demonstrate that our method exhibits robustness, accuracy, efficiency, and scalability. Furthermore, it can be readily adapted to new datasets. We are confident that our work holds the potential to significantly influence the tomato industry by effectively mitigating crop losses and enhancing crop yields.
BibTeX
@inproceedings{khan2023early,
  author = {Asim Khan and Umair Nawaz and Kshetrimayum Lochan and Lakmal D. Seneviratne and Irfan Hussain},
  title = {{Early and Accurate Detection of Tomato Leaf Diseases Using TomFormer}},
  booktitle = {ICAR},
  year = {2023},
  url = {https://openreview.net/forum?id=wq5Pi05dfG},
}

Loading related content...

Figure