Abstract
Robotic grasping of unknown objects with non-uniform mass distributions remains a significant challenge for vision-only systems. This paper presents a closed-loop grasping framework that integrates visual perception with tactile and torque sensing. By detecting slips in real-time and employing a novel LSTM-based regrasp planner to estimate an object's center of mass (CoM), the system achieves an 80% success rate on unknown objects, significantly outperforming state-of-the-art open-loop methods.
Introduction
Modern robotic systems often rely on "open-loop" visual grasp planning. While effective for objects with uniform density, these systems struggle with "top-heavy" tools like hammers or axes. Because vision cannot perceive internal mass distribution, initial grasp attempts often result in rotational slips due to unexpected torques. To address this, we propose a multi-modal system that "feels" the slip and iteratively adjusts the grasp point toward the object's true center of mass.
System Architecture
The proposed pipeline transitions from initial visual estimation to active tactile refinement.
- Initial Grasping: An RGB-D camera segments the target object. An Antipodal Grasp Sampler then identifies an initial grasp pose based on geometric heuristics.
- Slip Detection: During the lifting phase, the system monitors 4x4 tactile grids (taxels) and joint torque sensors. A classifier (comparing SVM and LSTM architectures) identifies the type of slip: clockwise, counter-clockwise, or translational.
- Regrasp Planning: Upon slip detection, a Regrasp Planner calculates a "regrasp ratio" ($\mu$). This ratio determines the optimal displacement along the object's axis to align the gripper with the CoM.
Regrasp Strategy
The core innovation lies in treating a slip not as a failure, but as informative data. As shown in the hammer example below, the direction and magnitude of the slip provide cues about the CoM location.
The regrasp planner utilizes an LSTM network to process temporal sequences of tactile and torque data. By predicting the regrasp ratio $\mu$, the robot can move its gripper to a more stable pose before re-attempting the lift.
Experimental Results
The system was evaluated using a Franka Emika robot on a dataset of 19 objects, including tools and modular LEGO models with hidden weights.
- Slip Classification: The LSTM-based classifier achieved 76.88% accuracy in identifying slip types on unknown objects.
- Grasping Performance: The closed-loop system reached a mean success rate of 80.0%. In contrast, the vision-only Dex-Net 4.0 achieved only 49.0%.
- Robustness: For highly asymmetric objects like heavy hammers, the success rate improved from 0% (vision-only) to 80% with the regrasp planner.
Conclusion
Integrating tactile feedback into the grasping loop allows robots to handle objects with complex physical properties that are invisible to cameras. Future work will focus on expanding the regrasp planner to support multi-dimensional adjustments and higher-resolution tactile sensing to further improve robustness across diverse object geometries.
Summary updated