Real-Time Grasp Planning for Multi-Fingered Hands by Finger Splitting

Yongxiang Fan, Te Tang, Hsien-Chung Lin, Masayoshi Tomizuka

Abstract Grasp planning for multi-fingered hands is computationally expensive due to the joint-contact coupling, surface nonlinearities and high dimensionality, thus is generally not affordable for real-time implementations. Traditional planning methods by optimization, sampling or learning work well in planning for parallel grippers but remain challenging for multi-fingered hands. This paper proposes a strategy called finger splitting, to plan precision grasps for multi-fingered hands starting from optimal parallel grasps. The finger splitting is optimized by a dual-stage iterative optimization including a contact point optimization (CPO) and a palm pose optimization (PPO), to gradually split fingers and adjust both the contact points and the palm pose. The dual-stage optimization is able to consider both the object grasp quality and hand manipulability, address the nonlinearities and coupling, and achieve efficient convergence within one second. Simulation results demonstrate the effectiveness of the proposed approach.

Introduction

Planning dexterous grasps for multi-fingered robotic hands is a significant challenge in robotics, often referred to as a "curse of dimensionality." Unlike simple parallel-jaw grippers, multi-fingered hands require accounting for complex joint-contact coupling, nonlinear object surfaces, and a vast search space for finger configurations. Traditional optimization or sampling-based methods frequently require minutes to find a stable grasp, which is impractical for real-time applications in dynamic environments.

Problem setup for multi-fingered grasping, defining the palm pose P, joint configurations q_i, and contact points c_i on the object O.

This paper introduces "finger splitting," a novel strategy designed to bridge the gap between the simplicity of parallel grasping and the dexterity of multi-fingered manipulation. By leveraging optimal parallel-jaw grasps as a starting point, the proposed method achieves high-quality precision grasps in under one second.

Methodology

The core of the proposed approach is the decoupling of the complex grasp planning problem into more manageable, high-speed optimization stages.

Finger Splitting Strategy

Instead of initiating a search from scratch, the algorithm initializes the multi-fingered hand (e.g., a 3-fingered Barrett hand) by grouping its fingers into two clusters that mimic a simple, optimal parallel grasp. This "seed" grasp provides a strong prior that drastically narrows the search space.

The finger splitting process: (a) initialization from a parallel grasp, (b) intermediate finger splitting, and (c) final precision grasp.

Dual-Stage Iterative Optimization

The optimization then proceeds through two alternating stages:

  1. Contact Point Optimization (CPO): While holding the palm position steady, the fingers are "split" and allowed to slide along the object's surface. The goal is to maximize grasp quality—defined by the triangle area of contacts—and hand manipulability. A "tangent space searching" technique is employed to move fingers efficiently before projecting them back onto the actual object surface, avoiding the need for complex analytical surface modeling.
  2. Palm Pose Optimization (PPO): With the contact points fixed, the palm's position and orientation are adjusted. This stage improves the hand's internal configuration and reachability, ensuring the planned contacts are feasible for the robot's kinematics.

To maintain real-time performance, the system utilizes supervoxel-based collision detection and a customized gradient projection method.

Results and Evaluation

The algorithm was evaluated using a variety of 3D meshes, including an apple, a screwdriver, a mug, and complex toys like the Stanford Bunny, with vertex counts ranging from 100 to 150,000. Initial parallel grasps were sourced from frameworks such as Dex-Net and Iterative Surface Fitting (ISF).

Key results include:

  • Computational Efficiency: The average total optimization time was 0.58 seconds (0.15s for preprocessing and 0.43s for optimization). This represents a significant improvement over previous state-of-the-art methods like HFTS, which can take up to 29 seconds.
  • Geometric Robustness: The system successfully planned stable grasps for non-convex surfaces where initial parallel grasps were infeasible.
  • Physical Stability: Simulations in the MuJoCo physics engine confirmed that the "split" grasps were stable enough to lift objects under gravity, outperforming the initial parallel-style configurations.

Discussion and Conclusion

The "finger splitting" strategy demonstrates that a good 2-point grasp is a powerful heuristic for multi-point grasping. By decoupling the optimization into CPO and PPO, the authors transformed an intractable problem into a real-time solution.

Future work will focus on enhancing collision handling through potential fields, allowing fingers to navigate around obstacles. Additionally, the authors plan to adapt the algorithm for hands with coupled joints and integrate raw point cloud data from depth cameras to improve real-world deployment.

Summary updated

Abstract

Grasp planning for multi-fingered hands is computationally expensive due to the joint-contact coupling, surface nonlinearities and high dimensionality, thus is generally not affordable for real-time implementations. Traditional planning methods by optimization, sampling or learning work well in planning for parallel grippers but remain challenging for multi-fingered hands. This paper proposes a strategy called finger splitting, to plan precision grasps for multi-fingered hands starting from optimal parallel grasps. The finger splitting is optimized by a dual-stage iterative optimization including a contact point optimization (CPO) and a palm pose optimization (PPO), to gradually split fingers and adjust both the contact points and the palm pose. The dual-stage optimization is able to consider both the object grasp quality and hand manipulability, address the nonlinearities and coupling, and achieve efficient convergence within one second. Simulation results demonstrate the effectiveness of the proposed approach. The simulation video is available at: http://me.berkeley.edu/\%7Eyongxiangfan/IROS2018/fingersplit.html
BibTeX
@inproceedings{fan2018real,
  author = {Yongxiang Fan and Te Tang and Hsien-Chung Lin and Masayoshi Tomizuka},
  title = {{Real-Time Grasp Planning for Multi-Fingered Hands by Finger Splitting}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2018},
  url = {https://openreview.net/forum?id=JSvZk0_E0DI},
}
Figure