FunGrasp: Functional Grasping for Diverse Dexterous Hands

Linyi Huang, Hui Zhang, Zijian Wu, Sammy Christen, Jie Song

Abstract

Functional grasping is essential for humans to perform specific tasks, such as grasping scissors by the finger holes to cut materials or by the blade to safely hand them over. Enabling dexterous robot hands with functional grasping capabilities is crucial for their deployment to accomplish diverse real-world tasks. Recent research in dexterous grasping, however, often focuses on power grasps while overlooking task- and object-specific functional grasping poses. In this paper, we introduce FunGrasp, a system that enables functional dexterous grasping across various robot hands and performs one-shot transfer to unseen objects. Given a single RGBD image of functional human grasping, our system estimates the hand pose and transfers it to different robotic hands via a human-to-robot (H2R) grasp retargeting module. Guided by the retargeted grasping poses, a policy is trained through reinforcement learning in simulation for dynamic grasping control. To achieve robust sim-to-real transfer, we employ several techniques including privileged learning, system identification, domain randomization, and gravity compensation. In our experiments, we demonstrate that our system enables diverse functional grasping of unseen objects using single RGBD images, and can be successfully deployed across various dexterous robot hands.

Introduction

In robotic manipulation, "dexterous grasping" has traditionally prioritized stability—simply holding an object without dropping it, often referred to as a "power grasp." However, human interaction with the world is fundamentally functional. We hold a mug by the handle to drink, but by the body to wash it; we hold scissors by the loops to cut, but by the blades to hand them to someone else.

Teaching robots this level of nuance is challenging due to morphology mismatches between human and robot hands, the difficulty of generalizing to unseen object shapes, and the significant sim-to-real gap. FunGrasp addresses these challenges by using human grasp images as priors to guide robotic hands, regardless of their specific design.

Methodology

Overview of the FunGrasp system showing the pipeline from human RGBD input to multi-robot execution.

The FunGrasp pipeline consists of three distinct stages designed to bridge the gap between human intent and robotic execution:

1. Static Functional Grasp Retargeting (H2R)

The system starts with a single RGBD image of a human performing a functional grasp. Instead of simply copying joint angles—which fails due to differing hand morphologies—FunGrasp optimizes the robot's pose to match the contact points and link directions of the human hand. This approach ensures that the functional intent (where the fingers touch the object) is preserved across different robotic platforms, such as the 16-DoF Allegro hand or the 6-DoF Inspire hand.

2. Dynamic Dexterous Grasping (RL)

Once a static target pose is established, a Reinforcement Learning (RL) policy is trained to manage the dynamic approach. The reward function incentivizes the robot to reach target joint positions while specifically rewarding "human-likeness" in posture and the accuracy of contact points.

3. Sim-to-Real Transfer

To achieve robust real-world deployment, the system employs several advanced techniques:

  • Privileged Learning: A "teacher" policy trained with perfect simulation data (e.g., exact contact forces) guides a "student" policy that uses only real-world sensors (proprioception).
  • System Identification: Simulation parameters are calibrated to match the real robot's joint stiffness and damping.
  • Gravity Compensation: The system calculates and counteracts the weight of the robot's fingers in real-time to improve precision.

Experimental Results

A dynamic functional grasp sequence (1-3) performed by the Allegro hand on a pair of scissors.

FunGrasp was validated using the DexYCB dataset and real-world tests on 20 completely unseen objects, including complex items like a hammer and a flower spray bottle.

  • One-Shot Success: The system achieved a 74% average success rate on unseen objects.
  • Cross-Platform Versatility: FunGrasp was successfully deployed on four different robotic hands (Shadow, Faive, Allegro, and Inspire) with minimal tuning, proving that prioritizing contact points over joint angles is key to cross-morphology transfer.
  • Robustness: The RL-based controller adapted to dynamic disturbances in real-time. Ablation studies highlighted the necessity of System Identification; without it, the real-world success rate plummeted from 75% to just 38%.

Conclusion and Future Work

FunGrasp demonstrates that functional grasping can be achieved through a combination of contact-based retargeting and robust RL policies. While the current system requires a known 3D mesh of the object for pose estimation, future work aims to develop an integrated model that operates directly from raw image inputs, further simplifying deployment in unstructured home environments.

Summary updated

Abstract

Functional grasping is essential for humans to perform specific tasks, such as grasping scissors by the finger holes to cut materials or by the blade to safely hand them over. Enabling dexterous robot hands with functional grasping capabilities is crucial for their deployment to accomplish diverse real-world tasks. Recent research in dexterous grasping, however, often focuses on power grasps while overlooking task- and object-specific functional grasping poses. In this paper, we introduce FunGrasp, a system that enables functional dexterous grasping across various robot hands and performs one-shot transfer to unseen objects. Given a single RGBD image of functional human grasping, our system estimates the hand pose and transfers it to different robotic hands via a human-to-robot (H2R) grasp retargeting module. Guided by the retargeted grasping poses, a policy is trained through reinforcement learning in simulation for dynamic grasping control. To achieve robust sim-to-real transfer, we employ several techniques including privileged learning, system identification, domain randomization, and gravity compensation. In our experiments, we demonstrate that our system enables diverse functional grasping of unseen objects using single RGBD images, and can be successfully deployed across various dexterous robot hands. The significance of the components is validated through comprehensive ablation studies. Project page: https://hly-123.github.io/FunGrasp/ .
BibTeX
@inproceedings{huang2025fungrasp,
  author = {Linyi Huang and Hui Zhang and Zijian Wu and Sammy Christen and Jie Song},
  title = {{FunGrasp: Functional Grasping for Diverse Dexterous Hands}},
  booktitle = {Dexterous Manipulation @ CoRL},
  year = {2025},
  url = {https://openreview.net/forum?id=gYQPuUdw45},
}
Figure