Abstract
Tomato leaf diseases pose a significant challenge to global agriculture, often resulting in substantial reductions in crop productivity. Timely and precise identification is crucial for implementing effective disease management strategies. This paper introduces TomFormer, a novel transformer-based fusion model designed for the early detection of tomato leaf diseases. By combining the local feature-extraction capabilities of Convolutional Neural Networks (CNNs) with the global context-awareness of Vision Transformers, TomFormer achieves state-of-the-art results. Evaluated across three datasets—KUTomaDATA, PlantDoc, and PlantVillage—the model achieved mean average precision (mAP) scores of 87%, 81%, and 83%, respectively. Furthermore, the model was successfully integrated into the Hello Stretch robot platform to demonstrate the potential for real-time, autonomous greenhouse diagnosis.
Introduction
Tomato crops are a global staple, yet they remain highly susceptible to a variety of pathogens that can devastate yields. For farmers, the difference between a minor intervention and a total crop loss often depends on the speed of detection. While deep learning has revolutionized plant pathology, two primary hurdles remain: the architectural limitations of traditional models and the deployment gap between laboratory research and field application.
Traditional CNNs excel at identifying local textures, such as specific lesions, but often struggle to understand the spatial relationships between symptoms across an entire leaf. Conversely, standard Transformers require significant data to learn local structures. TomFormer addresses this by employing a hybrid architecture. Additionally, to move beyond theoretical modeling, this research integrates the system into a mobile robotic platform capable of navigating real-world greenhouse environments.
Methodology
The TomFormer Architecture
TomFormer utilizes a "fusion" backbone designed to capture the best of both worlds in computer vision:
- Hybrid Backbone: The model uses a CNN block to extract low-level features (textures and edges) and a Transformer encoder to capture long-range dependencies. This allows the AI to analyze both tiny lesions and their distribution across the leaf structure simultaneously.
- Object Queries: Borrowing from the DEtection TRansformer (DETR) paradigm, TomFormer employs 20 learnable object queries. Instead of scanning an image with thousands of overlapping candidate boxes, the model uses these queries to locate and identify up to 15 leaves in a single pass.
- Direct Set Prediction: By treating detection as a direct set prediction problem and using the Hungarian algorithm for bipartite matching loss, the model eliminates the need for complex post-processing steps like Non-Maximum Suppression (NMS).
Robotic Integration
The model was deployed on the Hello Stretch robot, a mobile manipulator equipped with an Intel RealSense RGB-D camera and a Jetson Nano computer. This integration allows the system to navigate indoor greenhouse environments, capturing high-resolution images from various angles and providing immediate feedback to growers.
Experimental Evaluation
Datasets
The study evaluated TomFormer on three distinct datasets to test robustness and scalability:
- KUTomaDATA: A proprietary dataset of 939 images collected via mobile devices in a greenhouse in Abu Dhabi, UAE. It covers eight classes, including healthy leaves and diseases like Late Blight and Mosaic Virus.
- PlantDoc: A public dataset containing 2,598 data points across 13 plant species, providing a test for cross-species generalization.
- PlantVillage: A benchmark dataset featuring over 50,000 images captured under controlled laboratory conditions.
Performance and Results
TomFormer consistently outperformed established models, including YOLOS, DETR, ViT, and Swin Transformer.
- Accuracy: The model achieved an mAP of 87% on KUTomaDATA, 81% on PlantDoc, and 83% on PlantVillage.
- Efficiency: With only 1.30 million parameters, TomFormer is significantly more lightweight than many contemporary transformers, making it ideal for edge deployment on robotic platforms.
- Class-Specific Success: The model was particularly effective at identifying healthy leaves (92% mAP) and bacterial spots (90% mAP). It faced more challenges with "Early blight" (80% mAP) due to its visual similarity to other fungal infections.
Discussion and Limitations
A key insight from the study is that the model's performance on the KUTomaDATA dataset (87% mAP) is particularly significant. Unlike the uniform backgrounds of the PlantVillage dataset, KUTomaDATA reflects the "messy" and variable lighting conditions of a functional greenhouse.
However, some limitations remain. The paper notes a high inference latency in the current pipeline which may require further optimization for high-speed robotic movement. Additionally, while the results are promising, the KUTomaDATA dataset is relatively small. Future work will focus on expanding the training data to include more growth stages and diverse environmental conditions.
Conclusion
TomFormer represents a significant advancement in autonomous agricultural monitoring. By fusing CNN and Transformer architectures, the model provides a robust, efficient, and scalable solution for tomato leaf disease detection. When paired with mobile robotics, this technology holds the potential to significantly mitigate crop losses and enhance global food security through early, automated intervention.
Summary updated