Task-Oriented Dexterous Hand Pose Synthesis Using Differentiable Grasp Wrench Boundary Estimator

Jiayi Chen, Yuxing Chen, Jialiang Zhang, He WangIROS 2024 · OpenReview

TL;DR

This paper introduces a novel framework for generating dexterous hand poses that are specifically tailored to perform tasks, going beyond traditional force-closure grasps to include non-prehensile actions like turning knobs. By developing a fast, differentiable method to estimate the Grasp Wrench Space (GWS) and a new task-oriented objective function, they enable gradient-based optimization to synthesize effective hand poses without human demonstrations, achieving high success rates in simulation and real-world tests.

What Problem Does This Paper Address?

Robotic dexterous hands hold immense potential for complex manipulation, but current methods for generating hand poses are often limited. Many existing approaches focus exclusively on "force-closure grasps," which are grasps that can resist external forces from any direction. While useful, this is overly restrictive for many real-world tasks, such as turning a knob, pressing a button, or even pushing an object, where precise force resistance in all directions isn't necessary or even possible. These "non-prehensile" or "non-force-closure" tasks require hand poses that can apply specific wrenches (forces and torques) relevant to the task, not just any wrench. The challenge is to automatically generate these task-specific poses efficiently and accurately, especially for complex, multi-fingered hands.

Methods

The core of this work lies in two main innovations:

  • Differentiable Grasp Wrench Boundary (GWB) Estimator: The authors propose a new algorithm to estimate the boundary of the Grasp Wrench Space (GWS). The GWS represents all possible wrenches a hand can exert given its current pose and contact points. Their method is designed to be fast, accurate, and, crucially, differentiable. It achieves this by mapping arbitrary 6D unit directions to points on the GWB, leveraging geometric properties and an approximation technique to ensure dense sampling and avoid missing parts of the GWB. This is a significant improvement over traditional discretization-based methods that suffer from exponential complexity.
  • Task-Oriented Objective Function: Building on the GWB estimator, they introduce a novel objective function for gradient-based optimization. This function quantifies the "disparity" between the desired Task Wrench Space (TWS) – which defines the task's wrench requirements – and the estimated GWS. By minimizing this objective, the optimization process naturally guides the hand pose to conform to the task's specific wrench needs. The TWS is represented as a 6D hyper-spherical sector, allowing for flexibility in defining task requirements, including force-closure scenarios.

The entire synthesis pipeline is implemented using cuRobo, a GPU-accelerated motion planning library, enabling efficient, large-scale parallelization.

Datasets

The paper does not explicitly mention training datasets. For evaluation, they use:

  • YCB dataset [40]: Used for generating contact points on 49 objects for GWS estimator sanity checks.
  • DexGraspNet dataset assets: They leverage over 5700 object assets from DexGraspNet for large-scale force-closure grasp synthesis experiments.
  • Custom 10 tasks: For task-oriented synthesis, they created 10 diverse tasks involving turning, lifting, pushing, and pulling.

Key Insights and Ideas

  • Beyond Force-Closure: The most significant insight is that task-oriented manipulation requires moving beyond the strict definition of force-closure grasps. By focusing on the specific wrenches needed for a task (TWS), rather than just the ability to resist any wrench, they unlock a much broader range of manipulation capabilities for robots.
  • Differentiable GWS is Key: Making the GWS estimation differentiable is crucial. This allows gradient-based optimization methods, which are widely used and efficient, to directly learn and refine hand poses based on the GWS.
  • Mapping is More Efficient: The proposed mapping-based approach for GWB estimation is significantly faster and more scalable than traditional convex hull or discretization methods, especially for hands with many contacts.
  • Unified Framework: The ability to handle force-closure, non-force-closure, and non-prehensile tasks within a single, unified framework is a major step towards more versatile robotic manipulation.

Key Results

  • GWS Estimator Speed: The GWS boundary estimator runs in milliseconds, reportedly 200x faster than traditional discretization-based methods.
  • Task-Oriented Synthesis Success: Achieved a 72.6% success rate in simulation across 10 diverse tasks.
  • Real-World Validation: Demonstrated effectiveness in real-world experiments for 4 tasks (lid screwing, handle turning, knob screwing, drawer pulling).
  • DexGraspNet Comparison (Force-Closure): Synthesized 100,000 force-closure grasps 50x faster than DexGraspNet, with comparable grasp quality.
  • Efficiency: The entire synthesis pipeline is highly optimized, leveraging CUDA for large-scale parallelization.

Limitations and Future Work

  • Static Poses Only: The current framework generates static hand poses, not dynamic trajectories for complex manipulation tasks like in-hand reorientation.
  • Environment Interaction Not Modeled: The model does not explicitly account for contact between the object and its environment.
  • TWS Parameterization: The use of a hyper-spherical sector for TWS might not be universally suitable for all tasks.

Future work could explore generating dynamic trajectories, incorporating environmental interactions, and investigating more flexible TWS parameterizations for complex, long-horizon manipulation tasks. The authors also suggest that incorporating close-loop tactile feedback could further improve performance, especially when the finger misses the object by a small margin.

Summary updated

Abstract

This work tackles the problem of task-oriented dexterous hand pose synthesis, which involves generating a static hand pose capable of applying a task-specific set of wrenches to manipulate objects. Unlike previous approaches that focus solely on force-closure grasps, which are unsuitable for non-prehensile manipulation tasks (e.g., turning a knob or pressing a button), we introduce a unified framework covering force-closure grasps, non-force-closure grasps, and a variety of non-prehensile poses. Our key idea is a novel optimization objective quantifying the disparity between the Task Wrench Space (TWS, the desired wrenches predefined as a task prior) and the Grasp Wrench Space (GWS, the achievable wrenches computed from the current hand pose). By minimizing this objective, gradient-based optimization algorithms can synthe-size task-oriented hand poses without additional human demonstrations. Our specific contributions include 1) a fast, accurate, and differentiable technique for estimating the GWS boundary; 2) a task-oriented objective function based on the disparity between the estimated GWS boundary and the provided TWS boundary; and 3) an efficient implementation of the synthesis pipeline that leverages CUDA accelerations and supports large-scale parallelization. Experimental results on 10 diverse tasks demonstrate a 72.6% success rate in simulation. Furthermore, real-world validation for 4 tasks confirms the effectiveness of synthesized poses for manipulation. Notably, despite being primarily tailored for task-oriented hand pose synthesis, our pipeline can generate force-closure grasps 50 times faster than DexGraspNet while maintaining comparable grasp quality. Project page: https://pku-epic.github.io/TaskDexGrasp/.
BibTeX
@inproceedings{chen2024task,
  author = {Jiayi Chen and Yuxing Chen and Jialiang Zhang and He Wang},
  title = {{Task-Oriented Dexterous Hand Pose Synthesis Using Differentiable Grasp Wrench Boundary Estimator}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2024},
  url = {https://openreview.net/forum?id=RqwIGdinyu},
}
Figure