TL;DR
DAPPT (Domain-Aware Policies from Procedural Trajectories) allows robots to learn complex tool-use tasks involving particles and deformables—like scooping sand or rolling dough—without the computational burden of simulating those materials during training. By learning to track parameterized "procedural trajectories" in simple rigid-body simulations, the framework produces a flexible policy that can be optimized for real-world physics zero-shot.
What Problem Does This Paper Address?
Reinforcement Learning (RL) has become a powerhouse for "sim-to-real" transfer in robotics, particularly for rigid-body manipulation. However, once you introduce "messy" materials like sand, water, or play-dough, the wheels fall off.
Simulating millions of sand particles or elasto-plastic materials is computationally expensive, making the billions of samples required for RL training practically intractable. Furthermore, the "sim-to-real gap" for these materials is massive; even if you could simulate them, the physics often wouldn't match the real world closely enough for the policy to transfer. The authors of DAPPT ask: Can we learn to use tools effectively without simulating the materials the tools are meant to manipulate?
Methods
The core innovation of DAPPT is the Domain-Aware Policy. Instead of training a robot to "scoop sand," they train it to "track a parameterized trajectory of a scoop."
- Procedural Trajectories: The authors define tool motions algorithmically. For example, a scooping motion is a 6D curve defined by parameters like insertion angle, depth, and speed.
- Domain-Aware RL: During training in a fast, rigid-body simulator (IsaacLab), the policy is given a set of domain parameters ($c$) as input. These parameters dictate which specific trajectory the robot should follow. By randomizing these parameters during training, the robot learns a versatile "family" of behaviors rather than a single fixed motion.
- Two-Stage MDP: The training is split into a "transport" stage (reaching and grasping the tool) and a "tracking" stage (executing the procedural motion).
- Teacher-Student Distillation: A state-based teacher policy is first trained with full access to the simulation's "privileged" information. This is then distilled into a vision-based student policy that uses point clouds and proprioception, making it deployable on a real robot.
- Post-Hoc Optimization: Since the particles weren't simulated during training, the authors find the "best" domain parameters by testing the trained policy in a high-fidelity simulator (Genesis) or the real world and selecting the parameters that yield the highest task performance (e.g., the most sand scooped).
Datasets
The paper does not rely on a standard static dataset. Instead, it utilizes:
- IsaacLab: For high-speed rigid-body RL training.
- Genesis: A high-fidelity simulator used for "sim-to-sim" evaluation, featuring Material Point Method (MPM) simulations for sand and deformables.
- Real-World Evaluation: Conducted on a 19-DOF system consisting of a Franka Research 3 arm and a 12-DOF RobotEra XHand1.
Key Insights and Ideas
- Decoupling Execution from Physics: The most striking idea is that the robot doesn't need to "understand" sand physics during the RL phase. It only needs to learn the motor skills required to follow a wide variety of potential tool paths. The "understanding" of which path works best is deferred to a simple search over the domain parameters later.
- Domain Awareness as Robustness: By making the policy "domain-aware," the authors create a control behavior that is more stable. Ablation studies showed that a "fixed-domain" policy (trained on only one trajectory) failed significantly more often when transferred to complex physics, as it couldn't adapt to the new resistance forces of the sand or dough.
- Procedural > Demonstrations: While human demonstrations are often used to seed RL, they are fixed. Procedural trajectories allow the system to explore a mathematical space of motions, ensuring the robot can handle various initializations and tool poses.
Key Results
The researchers evaluated DAPPT on six challenging tasks: sand scooping, rod-rolling dough, sand scraping, mouse-wheel scrolling, dispenser pressing, and spray triggering.
- Performance Boost: Selecting the "Best" domain parameters improved task metrics by 1.5x to 3.8x compared to using average parameters.
- Sim-to-Sim: In high-fidelity particle simulations, DAPPT crushed the "SimToolReal" baseline, which achieved near-zero success on tasks like scraping and rolling because it lacked the contact-rich trajectory tracking capabilities of DAPPT.
- Real-World Success: The system achieved a 100% motion success rate in real-world scooping and high task success across the board (e.g., 7.7g of sand scooped per trial).
- Zero-Shot Transfer: The policy moved from a rigid-body sim to a real-world 19-DOF hand-arm system without any fine-tuning on real data.
Limitations and Future Work
- Manual Design: Currently, the procedural trajectory functions and the choice of domain parameters are hand-engineered by humans. The authors suggest that future work could use LLMs or "coding agents" to automate this process.
- Generalization: The policy is trained for specific tool categories. While it handles variations in pose, it might not generalize to a radically different tool shape without retraining.
- Known Objects: The system assumes the object (tool) is known during training to define the trajectories. Expanding this to "any-object" tool use remains a frontier.
For more details, you can find the code at https://github.com/princeton-vl/DAPPT.
Summary updated