DAPPT: Domain-Aware Policies from Procedural Trajectories for Sim-to-Real Dexterous Tool Use

Beining Han, Jia DengarXiv 2026 · arXiv

TL;DR

DAPPT (Domain-Aware Policies from Procedural Trajectories) allows robots to learn complex tool-use tasks involving particles and deformables—like scooping sand or rolling dough—without the computational burden of simulating those materials during training. By learning to track parameterized "procedural trajectories" in simple rigid-body simulations, the framework produces a flexible policy that can be optimized for real-world physics zero-shot.

What Problem Does This Paper Address?

Reinforcement Learning (RL) has become a powerhouse for "sim-to-real" transfer in robotics, particularly for rigid-body manipulation. However, once you introduce "messy" materials like sand, water, or play-dough, the wheels fall off.

Simulating millions of sand particles or elasto-plastic materials is computationally expensive, making the billions of samples required for RL training practically intractable. Furthermore, the "sim-to-real gap" for these materials is massive; even if you could simulate them, the physics often wouldn't match the real world closely enough for the policy to transfer. The authors of DAPPT ask: Can we learn to use tools effectively without simulating the materials the tools are meant to manipulate?

Methods

The core innovation of DAPPT is the Domain-Aware Policy. Instead of training a robot to "scoop sand," they train it to "track a parameterized trajectory of a scoop."

  1. Procedural Trajectories: The authors define tool motions algorithmically. For example, a scooping motion is a 6D curve defined by parameters like insertion angle, depth, and speed.
  2. Domain-Aware RL: During training in a fast, rigid-body simulator (IsaacLab), the policy is given a set of domain parameters ($c$) as input. These parameters dictate which specific trajectory the robot should follow. By randomizing these parameters during training, the robot learns a versatile "family" of behaviors rather than a single fixed motion.
  3. Two-Stage MDP: The training is split into a "transport" stage (reaching and grasping the tool) and a "tracking" stage (executing the procedural motion).
  4. Teacher-Student Distillation: A state-based teacher policy is first trained with full access to the simulation's "privileged" information. This is then distilled into a vision-based student policy that uses point clouds and proprioception, making it deployable on a real robot.
  5. Post-Hoc Optimization: Since the particles weren't simulated during training, the authors find the "best" domain parameters by testing the trained policy in a high-fidelity simulator (Genesis) or the real world and selecting the parameters that yield the highest task performance (e.g., the most sand scooped).

Datasets

The paper does not rely on a standard static dataset. Instead, it utilizes:

  • IsaacLab: For high-speed rigid-body RL training.
  • Genesis: A high-fidelity simulator used for "sim-to-sim" evaluation, featuring Material Point Method (MPM) simulations for sand and deformables.
  • Real-World Evaluation: Conducted on a 19-DOF system consisting of a Franka Research 3 arm and a 12-DOF RobotEra XHand1.

Key Insights and Ideas

  • Decoupling Execution from Physics: The most striking idea is that the robot doesn't need to "understand" sand physics during the RL phase. It only needs to learn the motor skills required to follow a wide variety of potential tool paths. The "understanding" of which path works best is deferred to a simple search over the domain parameters later.
  • Domain Awareness as Robustness: By making the policy "domain-aware," the authors create a control behavior that is more stable. Ablation studies showed that a "fixed-domain" policy (trained on only one trajectory) failed significantly more often when transferred to complex physics, as it couldn't adapt to the new resistance forces of the sand or dough.
  • Procedural > Demonstrations: While human demonstrations are often used to seed RL, they are fixed. Procedural trajectories allow the system to explore a mathematical space of motions, ensuring the robot can handle various initializations and tool poses.

Key Results

The researchers evaluated DAPPT on six challenging tasks: sand scooping, rod-rolling dough, sand scraping, mouse-wheel scrolling, dispenser pressing, and spray triggering.

  • Performance Boost: Selecting the "Best" domain parameters improved task metrics by 1.5x to 3.8x compared to using average parameters.
  • Sim-to-Sim: In high-fidelity particle simulations, DAPPT crushed the "SimToolReal" baseline, which achieved near-zero success on tasks like scraping and rolling because it lacked the contact-rich trajectory tracking capabilities of DAPPT.
  • Real-World Success: The system achieved a 100% motion success rate in real-world scooping and high task success across the board (e.g., 7.7g of sand scooped per trial).
  • Zero-Shot Transfer: The policy moved from a rigid-body sim to a real-world 19-DOF hand-arm system without any fine-tuning on real data.

Limitations and Future Work

  • Manual Design: Currently, the procedural trajectory functions and the choice of domain parameters are hand-engineered by humans. The authors suggest that future work could use LLMs or "coding agents" to automate this process.
  • Generalization: The policy is trained for specific tool categories. While it handles variations in pose, it might not generalize to a radically different tool shape without retraining.
  • Known Objects: The system assumes the object (tool) is known during training to define the trajectories. Expanding this to "any-object" tool use remains a frontier.

For more details, you can find the code at https://github.com/princeton-vl/DAPPT.

Summary updated

Abstract

Sim-to-real RL has achieved impressive results in dexterous manipulation but remains limited to tasks with accurate rigid-body simulation. However, these methods cannot be applied to skills that involve deformable and particles, with which the simulation speed will become too slow for large-scale RL learning. We present DAPPT, a sim-to-real framework for end-to-end dexterous tool-use manipulation skills, e.g., sand scooping, play-dough rod-rolling, without simulating non-rigid materials. Our method learns a domain-aware RL policy by tracking procedural trajectories parameterized by domain parameters, and distills it into an end-to-end domain-aware sensorimotor policy. Furthermore, explicit domain represen-tation enables fast selection of domain parameters that can maximize downstream task performance. We evaluate DAPPT end-to-end on six dexterous manipulation tasks, including sand scooping, play-dough rod-rolling, sand scraping, mouse-wheel scrolling, dispenser pressing, and spray triggering. Our method achieves 1.5-3.8x in task metrics through the best domain parameter and transfers zero-shot to the real world.
BibTeX
@misc{han2026dappt,
  author = {Beining Han and Jia Deng},
  title = {{DAPPT: Domain-Aware Policies from Procedural Trajectories for Sim-to-Real Dexterous Tool Use}},
  year = {2026},
  eprint = {2611.20202},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20202},
}
Figure