Real-Time Grasp Planning for Multi-Fingered Hands by Finger Splitting

Yongxiang Fan, Te Tang, Hsien-Chung Lin, Masayoshi Tomizuka

TL;DR

This paper introduces "finger splitting," a novel strategy that plans complex multi-fingered robotic grasps by starting from simple, optimal parallel-jaw grasps. By decoupling the optimization into two high-speed stages—Contact Point Optimization (CPO) and Palm Pose Optimization (PPO)—the system achieves high-quality precision grasps in under one second, making real-time multi-fingered manipulation feasible.

What Problem Does This Paper Address?

While parallel-jaw grippers (like the ones on many factory robots) are easy to plan for, they lack the dexterity of multi-fingered hands. However, planning for multi-fingered hands is a "curse of dimensionality" nightmare. You have to account for joint-contact coupling, nonlinear object surfaces, and a massive search space for finger configurations.

Traditional optimization or sampling methods often take minutes to find a stable grasp, which is far too slow for dynamic, real-world environments. The authors aim to bridge this gap: how can we take the "easy" knowledge we have about parallel grasping and transfer it to complex, multi-fingered hands in real-time?

Methods

The core innovation is the Finger Splitting strategy. Instead of starting from scratch, the algorithm initializes a multi-fingered hand (like the 3-fingered Barrett hand used here) by grouping its fingers into two clusters that mimic a simple parallel grasp.

The optimization then proceeds in two iterative stages:

  1. Contact Point Optimization (CPO): The palm position is held steady while the fingers "split" and slide along the object's surface to maximize grasp quality (triangle area of contacts) and hand manipulability. This uses a "tangent space searching" technique to move fingers efficiently before projecting them back onto the actual object surface.
  2. Palm Pose Optimization (PPO): The contact points are held steady while the palm's position and orientation are adjusted to improve the hand's internal configuration and reachability.

To keep things fast, they use a supervoxel-based collision detection method and a customized gradient projection that avoids the need for complex analytical modeling of every object surface.

Datasets

The authors didn't introduce a new training dataset, but they utilized existing frameworks for the "seed" grasps:

  • Dex-Net [9]: Used as a potential source for the initial optimal parallel grasps.
  • Iterative Surface Fitting (ISF) [10]: An alternative method to generate the starting parallel grasps.
  • Evaluation Objects: The algorithm was tested on a variety of 3D meshes including an apple, a screwdriver, a mug, and several toys (Doraemon, Hello Kitty, and the Stanford Bunny), ranging from 100 to 150,000 vertices.

Key Insights and Ideas

  • Knowledge Transfer: The most powerful idea here is that a good 2-point grasp is a fantastic "prior" for a multi-point grasp. It drastically narrows the search space.
  • Sliding as Optimization: During CPO, the fingers essentially "slide" over the object's surface. This behavior allows the hand to naturally adapt to the local geometry without needing a global map of the object.
  • Decoupling Works: By alternating between fixing the palm and fixing the contacts, the authors turned one massive, intractable optimization problem into two smaller, much faster ones.
  • Real-Time Feasibility: By achieving convergence in under a second, this method moves multi-fingered grasping out of the realm of "offline simulation" and into "online execution."

Key Results

  • Speed: The average total optimization time was 0.58 seconds (0.15s for preprocessing and 0.43s for optimization). This is significantly faster than previous state-of-the-art methods like HFTS (~17–29s) or analytical fitting (~23s).
  • Success across Geometry: The system successfully planned grasps for objects with complex, non-convex surfaces (like the Stanford Bunny) where initial parallel grasps were actually infeasible.
  • Stability: Simulation in the MuJoCo physics engine confirmed that the "split" grasps were stable enough to lift objects under gravity, whereas the initial parallel-style grasps often failed.

Limitations and Future Work

  • Collision Handling: The current algorithm stops searching if a collision is detected. The authors suggest adding "potential fields" in the future to allow fingers to actively navigate around obstacles rather than just stopping.
  • Hand Complexity: While tested on a Barrett hand, the authors plan to adapt the algorithm for hands with coupled joints (where moving one joint affects another).
  • Point Cloud Input: The current version relies on mesh data; future iterations aim to work directly with raw point clouds from depth cameras for even better real-world integration.

Summary updated

Abstract

Grasp planning for multi-fingered hands is computationally expensive due to the joint-contact coupling, surface nonlinearities and high dimensionality, thus is generally not affordable for real-time implementations. Traditional planning methods by optimization, sampling or learning work well in planning for parallel grippers but remain challenging for multi-fingered hands. This paper proposes a strategy called finger splitting, to plan precision grasps for multi-fingered hands starting from optimal parallel grasps. The finger splitting is optimized by a dual-stage iterative optimization including a contact point optimization (CPO) and a palm pose optimization (PPO), to gradually split fingers and adjust both the contact points and the palm pose. The dual-stage optimization is able to consider both the object grasp quality and hand manipulability, address the nonlinearities and coupling, and achieve efficient convergence within one second. Simulation results demonstrate the effectiveness of the proposed approach. The simulation video is available at [1].
BibTeX
@inproceedings{fan2018real,
  author = {Yongxiang Fan and Te Tang and Hsien-Chung Lin and Masayoshi Tomizuka},
  title = {{Real-Time Grasp Planning for Multi-Fingered Hands by Finger Splitting}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2018},
  url = {https://openreview.net/forum?id=JSvZk0_E0DI},
}

Loading related content...

Figure