FruitTouch: A Perceptive Gripper for Gentle and Scalable Fruit Harvesting

Ruohan Zhang, Mohammad Amin Mirzaee, Wenzhen Yuan

Abstract

The automation of fruit harvesting has gained increasing significance in response to rising labor shortages. A sensorized gripper is a key component of this process, which must be compact enough for confined spaces, able to stably grasp diverse fruits, and provide reliable feedback on fruit conditions for efficient harvesting. To address this need, we propose FruitTouch, a compact gripper that integrates high-resolution, vision-based tactile sensing through an optimized optical design. This configuration accommodates a wide range of fruit sizes while maintaining low cost and mechanical simplicity. Tactile images captured by an embedded camera provide rich information for real-time force estimation, slip detection, and softness prediction. We validate the gripper in real-world fruit harvesting experiments, demonstrating robust grasp stability and effective damage prevention.

Introduction

Agricultural labor shortages are driving a significant push toward automated harvesting solutions. However, picking soft, vine-grown fruits such as cherry tomatoes and strawberries remains a formidable challenge for robotic systems. These fruits typically grow in dense, cluttered clusters and are highly susceptible to bruising. Existing robotic end-effectors often struggle with this task, falling into two extremes: purely mechanical grippers that lack feedback and frequently crush or drop fruit, or advanced tactile sensors that are too bulky and expensive for practical agricultural deployment.

Overview of the FruitTouch gripper and its tactile perception capabilities.

To bridge this gap, we introduce FruitTouch, a low-cost, compact robotic gripper designed specifically for delicate harvesting. FruitTouch provides high-resolution feedback similar to advanced vision-based tactile sensors like GelSight but in a slim, scalable form factor suitable for navigating dense foliage. By integrating a single-camera, vision-based tactile sensor across two fingers, it enables real-time 3D force estimation, slip detection, and in-hand ripeness assessment.

Design and Methodology

The core innovation of FruitTouch lies in its integrated mechanical and optical design. Unlike traditional vision-based tactile sensors that require a camera for each sensing surface, FruitTouch utilizes a single centrally mounted camera to monitor both jaws.

Optical and Mechanical Integration

Through a clever system of mirrors, the single camera simultaneously captures the internal deformation of two silicone gel gripping pads. This shared optical path effectively halves the hardware cost and bulk, allowing for thin, wedge-shaped fingers that can penetrate tight spaces between fruits. The design is also geometrically scalable, allowing the gripper to be adapted for various fruit sizes by adjusting the rack and gear mechanism.

Mechanical and optical design of the FruitTouch gripper, illustrating the single-camera mirror system and scalability.

Tactile Perception

The system processes RGB tactile images to enable several key perceptual capabilities:

  • 3D Force Estimation: An embedded Multi-Layer Perceptron (MLP) maps gel deformations to surface normals. Normal force is estimated linearly from the gripper's motor current, while shear force is calculated by visually tracking marker displacement on the gel using Helmholtz–Hodge decomposition.
  • Slip Detection: By comparing the velocity of the fruit's reconstructed surface texture against the velocity of the gel's internal markers, the system can detect "incipient slip"—micro-movements occurring just before a full drop. This allows for preemptive grip adjustment.
  • Softness Assessment: Mimicking human pickers, the system uses a pairwise comparison model. A lightweight neural network compares video clips of the gripper squeezing a fruit against reference clips to determine relative ripeness, which is more robust than absolute firmness measurements.

Experimental Evaluation

We evaluated FruitTouch using custom-generated datasets for geometry calibration, force estimation (using ~10,000 data points), and softness training (using silicone fruit replicas at varying hardness levels).

In quantitative tests, the gripper estimated normal force with a Mean Absolute Error (MAE) of 0.267 N and shear force with an MAE of 0.149 N. The slip classifier achieved an F1 score of 0.692, predicting slip an average of 0.11 seconds before gross motion occurred. The pairwise softness model reached 94.7% accuracy on replicas.

Real-world harvesting experiments were conducted using a UR5e robotic arm. Implementing a tactile-informed "Slip+Force control" strategy significantly improved performance. For cherry tomatoes, picking success increased from 58.3% (open-loop) to 100%. For strawberries, success rose from 12.5% to 75%, while effectively preventing bruising through low force variability.

Discussion and Conclusion

FruitTouch demonstrates that high-resolution tactile sensing can be made compact and cost-effective for agricultural use. The use of mirrors to share a single camera across multiple sensing surfaces is a key design choice that transitions vision-based tactile sensing from a bulky lab setup to a field-ready tool. Furthermore, the relative softness ranking approach proves highly generalizable across different crop types without constant recalibration.

Current limitations include a reliance on surface texture for softness prediction, which is less effective for smooth fruits like cherry tomatoes. Future work will focus on scaling the design for larger fruits, such as apples (~75 mm diameter), and deploying the system in more dynamic, uncontrolled agricultural environments.

Summary updated

Abstract

The automation of fruit harvesting has gained increasing significance in response to rising labor shortages. A sensorized gripper is a key component of this process, which must be compact enough for confined spaces, able to stably grasp diverse fruits, and provide reliable feedback on fruit conditions for efficient harvesting. To address this need, we propose FruitTouch, a compact gripper that integrates high-resolution, vision-based tactile sensing through an optimized optical design. This configuration accommodates a wide range of fruit sizes while maintaining low cost and mechanical simplicity. Tactile images captured by an embedded camera provide rich information for real-time force estimation, slip detection, and softness prediction. We validate the gripper in real-world fruit harvesting experiments, demonstrating robust grasp stability and effective damage prevention.
BibTeX
@article{zhang2026fruittouch,
  author = {Ruohan Zhang and Mohammad Amin Mirzaee and Wenzhen Yuan},
  title = {{FruitTouch: A Perceptive Gripper for Gentle and Scalable Fruit Harvesting}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2026},
  doi = {10.1109/lra.2026.3692042},
}
Figure