Vision-based judgment of tomato maturity under growth conditions

Xinzhong Wang, Hanping Mao, Han Xu, Jianjun YinAFRICAN 2011 · DOI

Abstract

To determine the picking time of tomato and design the control strategy for the harvesting robot, the judgment of tomato maturity under natural conditions is required. Tomato samples were collected based on the fruit growth conditions which were divided into five different stages: breakers, turning, pink, light-red, and red stages. The visible CCD camera VS-880HC was adopted to shoot visible images, while the near-infrared images at a wavelength of 810 nm were screened by MS-3100 multi-spectral camera. The variations of samples, about color features, were analyzed. The tests indicated that with the changes in maturity, the hue-mean of tomato decreased and the red-green color-difference image mean increased. The standard deviations of the hue-mean and red-green image mean were the largest values for tomato in the pink stage, but the intensity mean of the near-infrared image for tomato in the pink stage had the lowest value. Hue-mean and red-green color-difference image mean can be used as a criterion for the judgment of tomato maturity, and the tests indicated that the red-green mean method was more satisfactory than that of the hue-mean in the maturity recognition methods of tomato fruit with an accuracy of over 96%.

Introduction

Automating the harvest of delicate crops like tomatoes is a critical challenge in agricultural robotics. While identifying a tomato's location is a solved problem in many contexts, determining the optimal picking time—its maturity—remains difficult under field conditions. Most existing research relies on controlled laboratory environments with uniform illumination. However, in real-world greenhouses, tomatoes grow under variable sunlight, overlap with foliage, and ripen unevenly. This study aims to bridge the gap by developing a robust vision-based system capable of grading tomato maturity directly on the vine across five distinct stages: breakers, turning, pink, light-red, and red.

Methodology

The researchers utilized the Lycopersicon esculentum (Cooperation 907#) cultivar for their study. A dual-camera setup was employed to capture both visible light (RGB) using a standard CCD camera and near-infrared (NIR) data at 810 nm using a multi-spectral camera.

The image processing pipeline involved segmenting the tomato from its complex background. Otsu’s method was applied for automatic thresholding to create a binary mask of the fruit. Two primary mathematical models were evaluated for maturity judgment:

  1. HSI Model: Analyzing the "Hue-mean" (average color tint).
  2. RGB Model: Analyzing the "Red-Green color difference mean."
Image processing steps for tomato segmentation: (a) original image, (b) initial segmentation, (c) refined mask, (d) final segmented fruit.

The dataset consisted of 135 images taken under natural illumination in greenhouses. From these, 174 specific instances where the fruit axis was perpendicular to the camera were selected. A random subset of 100 samples was used for final statistical validation of the maturity models. To simplify the initial computer vision challenge, a white paper was sometimes placed behind the tomatoes to highlight the fruit's profile.

Schematic of the camera setup used to capture images at specific angles relative to the fruit.

Results and Discussion

The Pink Stage Paradox

A key finding was that the "Pink" stage is the most challenging to classify. It exhibited the highest standard deviation in both hue and red-green color features due to non-uniform ripening patterns. Interestingly, while NIR intensity does not change linearly with maturity, it reaches a distinct minimum during the pink stage, suggesting its potential as a specialized validator for this specific phase.

Performance Comparison

The Red-Green color difference mean proved more robust than Hue-mean for field conditions. It provided a more linear and reliable metric as the fruit matured.

  • Overall Accuracy: The Red-Green mean method achieved over 96% accuracy, compared to 93% for the Hue-mean method.
  • Stage-Specific Precision: The system was exceptionally accurate at identifying "Red" tomatoes (100%) and "Breakers" (95-100%).
  • Numerical Thresholds: Clear "middle points" for the Red-Green mean were established at 0, 23.5, 42.5, and 70 to distinguish the five stages.

Conclusion

This research demonstrates a high-accuracy vision system for judging tomato maturity in natural growth conditions. While the use of a white paper background in some instances helped simplify segmentation, future work must address fully autonomous segmentation in complex environments. Additionally, addressing uneven ripening caused by sunlight exposure through multi-angle viewing or 3D modeling will be essential for improving the "whole-fruit" maturity score in robotic harvesting applications.

Summary updated

Abstract

To determine the picking time of tomato and design the control strategy for the harvesting robot, the judgment of tomato maturity under natural conditions is required. Tomato samples were collected based on the fruit growth conditions which were divided into five different stages in this article: breakers, turning, pink, light-red, and red stages. The visible CCD camera VS-880HC was adopted to shoot visible images, while the near-infrared images at a wavelength of 810 nm were screened by MS- 3100 multi-spectral camera. The variations of samples, about color features, were analyzed. The tests indicated that with the changes in maturity, the hue-mean of tomato decreased and the red-green colordifference image mean increased. The standard deviations of the hue-mean and red-green image mean were the largest values for tomato in the pink stage, but the intensity mean of the near-infrared image for tomato in the pink stage had the lowest value. Hue-mean and red-green color-difference image mean can be used as a criterion for the judgment of tomato maturity, and the tests indicated that the redgreen mean method was more satisfactory than that of the hue-mean in the maturity recognition methods of tomato fruit with an accuracy of over 96%. The intermediate divisions of five different maturity stages, which were divided by red-green color-difference image mean, were 0, 23.5, 42.5 and 70. The judgment errors of the two methods are mainly caused by the recognition of tomatoes at the pink stage. Key words : Tomato, maturity, image, judgment.
BibTeX
@inproceedings{wang2011vision,
  author = {Xinzhong Wang and Hanping Mao and Han Xu and Jianjun Yin},
  title = {{Vision-based judgment of tomato maturity under growth conditions}},
  booktitle = {AFRICAN},
  year = {2011},
  doi = {10.5897/ajb11.015},
}
Figure