Utilizing Vision-Language Models for Detection of Leaf-Based Diseases in Tomatoes

James Blossom EleojoAAAI 2025 · DOI

TL;DR

This paper proposes using Vision-Language Models (VLMs) to automatically detect and classify tomato leaf diseases, addressing the limitations of manual inspection. By fine-tuning a pre-trained VLM on a large dataset of tomato leaf images and their corresponding disease annotations, the approach aims to improve accuracy and enable real-time monitoring for precision farming.

What Problem Does This Paper Address?

Tomato leaf diseases pose a significant threat to global food security and have substantial economic impacts. Early and accurate detection is crucial for improving crop yields. Traditional methods rely on human expertise, which is often inaccurate, time-consuming, and tiring. While computer vision and machine learning have been applied to plant disease detection, this paper specifically explores the enhanced capabilities of Vision-Language Models (VLMs) to leverage both visual and semantic information for more robust and accurate disease diagnosis.

Methods

The proposed approach combines computer vision and natural language processing (NLP) techniques within a multi-modal learning framework.

Computer Vision Component: * Image Pre-processing: Tomato leaf images are resized, normalized, and augmented to enhance dataset diversity and robustness. * Feature Extraction: A multi-stream convolutional neural network (MSCNN) is used to extract visual features, aiming for improved accuracy over single CNN approaches. * Image Embedding: Extracted visual features are embedded into a vector space to represent image characteristics.

Natural Language Component: * Text Pre-processing: Disease labels and descriptions are processed using NLP techniques like tokenization and stopword removal. * Text Embedding: The processed text is embedded into a vector space to capture semantic meaning.

Multi-modal Fusion: * Joint Embedding Space: Image and text embeddings are fused into a unified space to learn cross-modal relationships. * Vision-Language Model (VLM): A pre-trained VLM (e.g., CLIP, VisualBERT) is fine-tuned on this joint embedding space to learn disease-specific features. * Image-text Matching: The fine-tuned VLM predicts disease labels by matching image embeddings with text embeddings. * Disease Classification: Predicted labels are used to classify images into specific disease categories.

The paper highlights the uniqueness of this approach through its use of a multi-stream CNN and multi-modal fusion for increased accuracy.

Datasets

The paper mentions using a "very large dataset of tomato leaf images, healthy and diseased" for training. Specific details about the dataset's size, origin, or composition are not provided beyond this general description.

Key Insights and Ideas

  • Leveraging Multi-modality: Combining visual information from leaf images with textual descriptions of diseases offers a more comprehensive understanding than single-modal approaches.
  • VLM Potential for Agriculture: VLMs, known for tasks like image captioning and visual question-answering, show promise for practical applications in agriculture, such as automated disease diagnosis.
  • Zero-Shot Generalization: The evolution of VLMs is leading to improved zero-shot generalization capabilities, which can be beneficial for identifying novel or less common disease instances.
  • Enhanced Feature Extraction: Employing a multi-stream CNN for feature extraction is proposed as a method to boost detection accuracy.

Key Results

The paper does not present specific quantitative results or performance metrics in the provided text. It outlines the proposed approach and the evaluation criteria but does not detail the outcomes of experiments.

Limitations and Future Work

Limitations: * Dataset Quality: The performance of VLMs is limited by the quality of training data. * Misclassification: Diseases with similar symptoms can lead to misclassifications. * Image Quality Issues: Practical deployment can be hindered by variations in image quality, resolution, and lighting. * Computational Demands: VLMs can be computationally intensive.

Future Work: * Improving Model Performance: Further enhancements to VLM performance are suggested. * Exploring New Architectures: Investigating novel VLM architectures is a potential direction. * Real-World Integration: Integrating disease detection systems into real-world scenarios and mobile devices for accessibility and real-time detection is a key future goal.

Summary updated

BibTeX
@inproceedings{eleojo2025utilizing,
  author = {James Blossom Eleojo},
  title = {{Utilizing Vision-Language Models for Detection of Leaf-Based Diseases in Tomatoes}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i28.35327},
}
Figure