Abstract
To determine the picking time of tomato and design the control strategy for the harvesting robot, the judgment of tomato maturity under natural conditions is required. Tomato samples were collected based on the fruit growth conditions which were divided into five different stages: breakers, turning, pink, light-red, and red stages. The visible CCD camera VS-880HC was adopted to shoot visible images, while the near-infrared images at a wavelength of 810 nm were screened by MS-3100 multi-spectral camera. The variations of samples, about color features, were analyzed. The tests indicated that with the changes in maturity, the hue-mean of tomato decreased and the red-green color-difference image mean increased. The standard deviations of the hue-mean and red-green image mean were the largest values for tomato in the pink stage, but the intensity mean of the near-infrared image for tomato in the pink stage had the lowest value. Hue-mean and red-green color-difference image mean can be used as a criterion for the judgment of tomato maturity, and the tests indicated that the red-green mean method was more satisfactory than that of the hue-mean in the maturity recognition methods of tomato fruit with an accuracy of over 96%.
Introduction
Automating the harvest of delicate crops like tomatoes is a critical challenge in agricultural robotics. While identifying a tomato's location is a solved problem in many contexts, determining the optimal picking time—its maturity—remains difficult under field conditions. Most existing research relies on controlled laboratory environments with uniform illumination. However, in real-world greenhouses, tomatoes grow under variable sunlight, overlap with foliage, and ripen unevenly. This study aims to bridge the gap by developing a robust vision-based system capable of grading tomato maturity directly on the vine across five distinct stages: breakers, turning, pink, light-red, and red.
Methodology
The researchers utilized the Lycopersicon esculentum (Cooperation 907#) cultivar for their study. A dual-camera setup was employed to capture both visible light (RGB) using a standard CCD camera and near-infrared (NIR) data at 810 nm using a multi-spectral camera.
The image processing pipeline involved segmenting the tomato from its complex background. Otsu’s method was applied for automatic thresholding to create a binary mask of the fruit. Two primary mathematical models were evaluated for maturity judgment:
- HSI Model: Analyzing the "Hue-mean" (average color tint).
- RGB Model: Analyzing the "Red-Green color difference mean."
The dataset consisted of 135 images taken under natural illumination in greenhouses. From these, 174 specific instances where the fruit axis was perpendicular to the camera were selected. A random subset of 100 samples was used for final statistical validation of the maturity models. To simplify the initial computer vision challenge, a white paper was sometimes placed behind the tomatoes to highlight the fruit's profile.
Results and Discussion
The Pink Stage Paradox
A key finding was that the "Pink" stage is the most challenging to classify. It exhibited the highest standard deviation in both hue and red-green color features due to non-uniform ripening patterns. Interestingly, while NIR intensity does not change linearly with maturity, it reaches a distinct minimum during the pink stage, suggesting its potential as a specialized validator for this specific phase.
Performance Comparison
The Red-Green color difference mean proved more robust than Hue-mean for field conditions. It provided a more linear and reliable metric as the fruit matured.
- Overall Accuracy: The Red-Green mean method achieved over 96% accuracy, compared to 93% for the Hue-mean method.
- Stage-Specific Precision: The system was exceptionally accurate at identifying "Red" tomatoes (100%) and "Breakers" (95-100%).
- Numerical Thresholds: Clear "middle points" for the Red-Green mean were established at 0, 23.5, 42.5, and 70 to distinguish the five stages.
Conclusion
This research demonstrates a high-accuracy vision system for judging tomato maturity in natural growth conditions. While the use of a white paper background in some instances helped simplify segmentation, future work must address fully autonomous segmentation in complex environments. Additionally, addressing uneven ripening caused by sunlight exposure through multi-angle viewing or 3D modeling will be essential for improving the "whole-fruit" maturity score in robotic harvesting applications.
Summary updated