TL;DR
RoTipBot is a novel robotic system equipped with rotatable, vision-based tactile sensors designed to handle thin, flexible objects like paper, plastic sheets, and fabrics. Unlike traditional grippers that can only manipulate a single sheet at a time, RoTipBot can accurately count and grasp multiple layers simultaneously in a single grasp closure, operating up to three times faster than current state-of-the-art methods.
What Problem Does This Paper Address?
Manipulating thin, flexible objects—such as turning the page of a book, separating plastic wrap, or sorting fabrics—is a notoriously difficult task for robots. These objects present two main challenges:
- Perception Noise: Because they are so thin, stacked layers are nearly impossible to distinguish using standard cameras alone, leading to incomplete or noisy visual data.
- Deformability: Their tendency to bend, flex, or stick together requires highly dexterous and compliant handling to avoid tearing or double-feeding.
Current solutions typically rely on suction cups or soft two-finger grippers. However, suction cups fail when air gaps prevent a vacuum seal, and soft grippers are generally limited to singulating (picking up just one sheet at a time). RoTipBot addresses these limitations by introducing a system that can dynamically feed, count, and grasp multiple layers at once.
Methods
The core innovation lies in the RoTip sensor, a vision-based tactile sensor belonging to the GelSight family. It features a soft elastomer fingertip capable of omnidirectional contact sensing, paired with an external motor that allows for unconstrained, continuous active rotation.
The RoTipBot framework operates in three main phases:
- Vision-Guided Initialization: An RGB-D camera detects the corner of the object (aided by an ArUco marker) to generate a rough initial grasp pose.
- Tactile-Based Pose Adjustment: To ensure secure two-finger contact and compensate for visual errors, the robot touches the object. A fine-tuned DeepLabV3+ segmentation network processes the tactile image to find the contact area, and a RANSAC-based plane-fitting algorithm calculates the exact surface normal. The robot then adjusts its orientation to align perfectly with the object.
- Continuous Feeding and Counting: One finger remains passive to hold the stack, while the active finger rotates continuously to roll multiple layers into the gripper center. A YOLOv11 model monitors the tactile images in real-time, detecting paper edges as they pass over the sensor to count the layers. Simultaneously, the gripper dynamically adjusts its position to accommodate the growing thickness of the grasped stack.
Datasets
To train the contact detection and segmentation models, the authors collected a dataset of 392 tactile images with annotated ground-truth masks by interacting with printer paper and various textured fabrics at multiple angles. Using data augmentation techniques (flipping, rotation, cropping, and contrast adjustments), this dataset was expanded to 4,000 tactile images (split 70:30 for training and testing).
Key Insights and Ideas
- The Power of the Corner: Through Finite Element Method (FEM) simulations and beam theory, the authors proved that the corners of thin, flexible objects are the mathematically optimal grasping points. Squeezing along the angle bisector of a corner minimizes the elastic force exerted by the object, preventing damage and making it easier to separate layers.
- Mobilized Tactile Sensing: Traditional tactile sensors are static fixtures on a gripper. By making the sensing surface actively rotatable without angle constraints, the authors successfully bridged the gap between perception and dexterity, turning a sensor into an active manipulation tool.
Key Results
- Superior Perception: RoTip’s tactile plane estimation achieved an average error of just 1.51°, significantly outperforming vision-only methods (7.82°) and robot force-torque sensors (11.14°).
- 100% Contact Success: The tactile-based adjustment strategy achieved a 100% success rate for establishing secure two-finger contact.
- Speed and Efficiency: RoTipBot achieved a grasping success rate of over 90% across various tilt angles and operated up to 3× faster than state-of-the-art baselines like FlipBot.
- High Counting Accuracy: The system counted layers with 97% accuracy for printer paper, 94% for coated paper, and 98% for plastic sheets.
Limitations and Future Work
Despite its success, RoTipBot faces challenges with high-friction materials (like rough fabrics), where the friction between layers is higher than the friction between the sensor and the top layer, causing the stack to bunch up rather than separate cleanly. It also struggles with wrinkled or crumpled objects, which disrupt uniform force distribution.
In the future, the authors plan to miniaturize the RoTip sensor to fit multi-fingered dexterous hands and explore Reinforcement Learning (RL) to develop more generalized handling policies for highly unpredictable deformable materials.
Summary updated