Abstract
Training dexterous robot manipulation policies often requires labor-intensive reward engineering or complex kinematic retargeting of human demonstrations. This paper introduces a scalable alternative: using a predictive model of human motion to guide robot learning. By representing both humans and robots through a shared abstraction of 3D fingertip keypoints, a motion predictor trained on human datasets can be used zero-shot to provide dense tracking rewards. This approach enables robots to learn complex tasks like grasping, lifting, and opening cabinets without manual reward design, demonstrating successful cross-embodiment transfer across various robotic hands.
Introduction
The challenge of teaching robots dexterous manipulation lies in the "reward gap." Sparse rewards (e.g., "did the object move?") are easy to define but difficult to learn from, while dense rewards (step-by-step instructions) require significant engineering effort. While large-scale human video datasets exist, mapping human motion to robot joints (retargeting) is technically demanding and environment-specific. This work proposes bypassing these hurdles by focusing on fingertip trajectories—a "universal language" shared by humans and anthropomorphic robots.
Methodology: Human Motion Prediction
The core of the framework is the Human Motion Predictor ($\Pi_h$). This model is a causal transformer trained on the DexYCB dataset, which contains 1,000 trajectories of human hand-object interactions.
The predictor takes a history of 3D fingertip locations ($k^h$) and object point clouds ($o$) to forecast future fingertip positions ($\widehat{k}^h$). Because the model operates on 3D coordinates rather than joint angles, it can be applied directly to robotic hands that share a similar fingertip configuration, regardless of their internal degrees of freedom (DoF).
Reward Formulation and Policy Optimization
During reinforcement learning (RL) in a simulated environment (IsaacGym), the robot's policy $\pi$ is optimized using Proximal Policy Optimization (PPO). The learning process is guided by a composite reward function:
- Tracking Reward ($r_{track}$): The robot receives a reward for minimizing the Euclidean distance between its current fingertip positions ($k^r$) and the "human-like" positions predicted by the frozen $\Pi_h$.
- Task Reward ($r_{task}$): A sparse signal indicating task completion.
This setup allows the motion predictor to act as a "universal coach," providing a natural curriculum. For instance, in a drawer-opening task, the predictor naturally guides the hand toward the handle before encouraging a pulling motion, as observed in human data.
Experimental Results
The proposed method was evaluated across several challenging tasks, including Grasp and Lift, Lift and Throw, and Open Cabinet. Key findings include:
- Cross-Embodiment Transfer: The same human-trained predictor successfully guided three different robot hands: the Allegro (16-DoF), Xhand (12-DoF), and SVH (20-DoF).
- Performance: Success rates were comparable to "privileged" baselines that used manually tuned dense rewards.
- Generalization: The model provided useful guidance even for tasks not explicitly present in the training data, such as throwing objects.
Discussion and Limitations
While effective, the framework currently requires the robot to have an anthropomorphic form factor to utilize the fingertip keypoint abstraction. Future work is needed to extend this to non-humanoid grippers. Additionally, while the policies show promise in simulation, real-world deployment (sim-to-real) remains a challenge due to potential sensor noise and the requirement for accurate object point cloud reconstruction.
Conclusion
By leveraging human motion prediction as a dense reward signal, this work offers a path toward more scalable robot learning. Decoupling the human data from the RL training loop allows for a "train once, use anywhere" model that significantly reduces the burden of reward engineering for dexterous manipulation.
Summary updated