Abstract
Controlling high-degree-of-freedom (DoF) robotic hands remains a significant challenge due to the "curse of dimensionality." Traditional reinforcement learning methods often struggle with data efficiency and generalization because they couple the robot's dynamics with the environment. This paper introduces MoDex, a framework that mimics the biological principle of internal models by learning decoupled hand dynamics. By training a pair of neural networks—forward and inverse models—through random exploration, MoDex enables efficient bidirectional planning. When integrated with Large Language Models (LLMs) for heuristic generation, MoDex achieves state-of-the-art performance in dexterous tasks, demonstrating up to 100x better data efficiency than traditional model-free baselines.
Introduction
Robotic hands with high DoF, such as the 24-DoF Shadowhand or the 39-DoF Myohand, offer human-like dexterity but are notoriously difficult to control. Standard Deep Reinforcement Learning (DRL) approaches treat the robot and its environment as a single complex system, requiring an astronomical number of trials to learn even basic movements.
In contrast, neuroscience suggests that humans utilize an internal model—a mental representation of the body's dynamics—to predict limb movement and determine necessary motor commands. MoDex adopts this biological advantage, learning a task-agnostic "hand model" that can be "plugged in" to various external environments and tasks without retraining from scratch.
Methodology
The MoDex architecture is built upon three core components:
1. Neural Internal Models
MoDex begins by learning the hand's intrinsic physics through task-free random exploration. This results in two complementary models:
- Forward Model ($f_\theta$): Predicts future fingertip states based on the current state and action.
- Inverse Model ($g_\phi$): Maps a desired target state to a distribution of possible actions. Modeling this as a Gaussian distribution accounts for the redundancy in high-DoF systems where multiple joint configurations can achieve the same fingertip position.
2. Bidirectional Planning
Rather than relying on random shooting for trajectory optimization, MoDex employs a bidirectional strategy. The Inverse Model provides a high-quality initial "warm start" for action sequences, which is then iteratively refined by a Cross-Entropy Method (CEM) planner using the Forward Model. This approach significantly accelerates convergence compared to standard model-based planning.
3. Factorized Dynamics and LLM Integration
MoDex factorizes system dynamics into a fixed hand model and a task-specific external model (e.g., object dynamics). For high-level task specification, MoDex leverages Large Language Models (LLMs). By prompting an LLM (like GPT-4) to generate Python-based cost functions for specific gestures (e.g., "Rock & Roll"), the system can optimize low-level motor control to satisfy high-level semantic requests, bridging the gap between language and robotics.
Experimental Results
The framework was evaluated across diverse simulated environments (IsaacGym, Myosuite) and on real-world hardware (Xhand).
- Data Efficiency: In fingertip reaching tasks, MoDex achieved superior performance using only 1% to 10% of the data required by model-free methods like Soft Actor-Critic (SAC).
- High-DoF Success: On the 39-DoF Myohand—the most complex model tested—MoDex reached an 88% success rate in sequential reaching, whereas traditional synergy-based baselines failed entirely.
- Real-World Robustness: The system successfully performed in-hand rotation of various objects, including cubes, cylinders, and irregular shapes like an orange, within just 30 minutes of real-world rollout.
- Zero-Shot Transfer: Internal models learned for simple reaching were successfully applied to complex object manipulation without needing task-specific retraining.
Discussion and Limitations
While MoDex represents a leap in data-efficient dexterous control, several challenges remain. The current implementation relies on precise state tracking (e.g., joint positions or AprilTags), which is difficult to maintain in unstructured real-world environments. Furthermore, the use of simple MLP architectures for dynamics might be superseded by more expressive models, such as Diffusion Models, to better capture the multi-modal nature of contact physics. Future work will also explore the integration of tactile feedback to enhance the stability of in-hand manipulation for soft or slippery objects.
Summary updated