Handoff-Aware Randomization for Chaining Dexterous Skills

Robots with multi-fingered hands can now learn single contact-rich skills in simulation and run them on real hardware: grasp an object, rotate it in the hand, insert a peg. Long tasks need these skills run one after another, and this is where things break. When a grasp skill finishes, the object sits in the palm at some pose, with some fingers in contact. The next skill, say an in-hand rotation, was trained from a different set of starting configurations. On real hardware, unmodeled friction and contact stiffness shift where the first skill actually ends, so the second skill starts somewhere it has never seen. We propose to treat the starting states of each skill as a quantity to randomize in simulation, the same way physics parameters are randomized today. Each skill learns a map of the joint space of start states and physics parameters from which it reliably succeeds. That map is fitted to cover where its predecessors really end up under randomized physics. The same map then serves two purposes: it rewards predecessors for ending inside it, and it tells a high-level planner when a handoff is unsafe.

The opportunity comes from two observations that sit side by side in the literature but have not been combined. Domain randomization, meaning training under randomly varied simulator physics so the policy survives the real world, works much better when its distribution is learned rather than hand-set. A learned distribution reached 90% real gear insertion against 60% for broad uniform randomization in Flow-based Domain Randomization for Learning and Sequencing Robotic Skills. Hierarchies of skills, meanwhile, transfer to hardware only when randomization is split across levels: 73–90% real success against 0% for a single policy trained with randomization in Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Real. Neither line has addressed the boundary between two dexterous skills. That boundary is where the research file locates the "compositional mismatch" failure of modular skill systems.

Core Claim

Our hypothesis is that chains of separately trained dexterous skills mostly fail at handoffs, not inside individual skills. If so, the cheapest fix is to learn, per skill, a density over a compact handoff descriptor and the physics parameters together. This density should be shaped to cover the predecessors' real termination distribution, and used both as a training signal and as a run-time precondition. Under this hypothesis, a library of frozen dexterous skills can be recombined into new multi-stage tasks and deployed zero-shot on hardware. It should match per-task end-to-end fine-tuning while costing far fewer environment steps per new task.

There are three differences from the closest work. GoFlow learns a competence density over physics parameters for one skill and uses it only to trigger an inspection action. We put start states into the density and couple adjacent skills through it. R²S² (Unleashing Humanoid Reaching Potential via Real-World-Ready Skill Space) smooths transitions by distilling primitives into a shared latent space, but only for whole-body reaching with simple end-effectors and with hand-set randomization. DIPPER (Direct Preference Optimization for Primitive-Enabled Hierarchical RL) filters infeasible subgoals with the low-level value function. It has been tested only on 7–9-DoF arms without physics variation, and its authors flag high-dimensional subgoal spaces as unverified. Our handoff descriptor is our answer to that dimensionality problem: feasibility is judged over a dozen task-relevant quantities, not the full 24-DoF hand state.

Research Program

The skill library contains $K$ dexterous skills. Each is trained with reinforcement learning in Isaac-based simulation using privileged state, then distilled to vision and touch, following the teacher–student recipe of ADEPT. Planned skills are power grasp, in-hand reorientation to a target pose, regrasp, handover between hands, and insertion or placement.

At each skill boundary we define a handoff descriptor $h$. It contains the object pose in the palm frame, the set of fingers in contact, normal-force magnitudes, and object velocity. It deliberately excludes the full joint configuration. For skill $k$ we fit a normalizing flow, a density model built from invertible transforms that can represent multi-modal, correlated distributions. The flow $q_k(h, \xi)$ is defined over handoff descriptors $h$ and simulator physics parameters $\xi$ (friction, contact stiffness, mass, actuator gain and delay). We fit it by extending GoFlow's objective with one coverage term:

$$ \max_{q_k}\; \mathbb{E}_{q_k}\big[S_k(h,\xi)\big] + \alpha\, \mathcal{H}(q_k) - \beta\, \mathrm{KL}(q_k \,\|\, q_k^{\text{old}}) + \gamma\, \mathbb{E}_{(h,\xi)\sim D_{k-1}}\big[\log q_k(h,\xi)\big] $$

The symbols are defined as follows. $S_k(h,\xi)$ is the success probability of skill $k$ started from $h$ under physics $\xi$, estimated from rollouts. $\mathcal{H}$ is the entropy of the flow. The KL term against the previous iterate keeps training stable. $D_{k-1}$ is the set of terminal descriptors and physics draws actually produced by all skills that can precede $k$. The last term is the new ingredient: it pulls the competence map toward where predecessors really land rather than where a designer guessed they would.

Training alternates three steps. First, we roll out predecessors under randomized physics to refresh $D_{k-1}$. Second, we refit $q_k$. Third, we train skill $k$ from start states sampled from a mixture of $q_k$ and $D_{k-1}$, and give each predecessor a terminal bonus proportional to $\log q_{k}(h_T, \xi)$. This bonus pushes predecessors to end where successors are competent.

At deployment the high-level planner chooses the next skill and its termination target. It scores each candidate by the marginal density of the current descriptor under the successor's map, averaged over a belief about $\xi$ maintained by a recurrent estimator in the style of AdaptManip. If no successor's density clears a threshold, the planner calls the regrasp skill. The planner itself is a small learned policy over skill identities and termination targets. It is regularized by the density score in the way DIPPER regularizes with the low-level value function.

Low-level skills stay frozen after library training. This sides with the frozen-skill camp in the tension the file identifies against Skill-Critic and online residual tuning (Efficient Sim-to-real Transfer of Contact-Rich Manipulation Skills with Online Admittance Residual Learning). Our bet is that most of the benefit those papers get from adapting skills comes from repairing handoff mismatch, which we address at training time instead. We test this bet directly rather than assume it.

Baselines

The naive reference point is the chain of independently trained skills, each with standard uniform physics randomization and hand-designed start-state ranges, sequenced by a fixed script. It is the system a practitioner would build first. It shows how large the handoff problem is before any fix.

The strongest existing method for real multi-fingered contact-rich tasks is ADEPT: pre-train on object reposing, then post-train each downstream task with distillation, critic warm-up, and conservative PPO. It reports 8/10 real success on the asymmetric FMB peg and 6/10 on the long-horizon dish-rack task, with post-training reaching success in about 3 billion environment steps (ADEPT). We run ADEPT on each of our multi-stage tasks with the same pre-training checkpoint, the same randomization ranges, the same student sensors, and the same hardware. We report its environment-step cost per new task alongside success. Beating it means matching or exceeding its real success on held-out chains with no per-chain training of low-level skills. The new-chain compute has to stay at least an order of magnitude below ADEPT's per-task post-training budget.

The strongest hierarchical competitor is a DIPPER-style planner over the same frozen skill library. Its subgoals are filtered by each skill's value function, trained under the same physics randomization and averaged over it. This baseline isolates whether a density over handoff states and physics buys anything beyond a robust value function. It gets the same rollout budget we spend fitting flows.

We also run GoFlow applied per skill over physics only, with hand-set start states and no coverage term. Finally, two ablations each remove one of our ingredients. The first removes the coverage term (the $\gamma$ term) and the predecessor bonus. The second replaces the compact descriptor with full hand-and-object state.

Why not just…?

Why not just fine-tune one pre-trained dexterous policy per task, as ADEPT does? On a fixed task this is a serious competitor and may win on peak success. But it pays billions of environment steps for every new task. It also gives no way to reuse work across tasks, and Bi-DexHands shows that pushing many dexterous tasks into one policy collapses: multi-task performance "craters" and meta-learning scores sit near zero (Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning). Our comparison is therefore on held-out chains built from library skills, at matched new-task compute. If ADEPT at a tenth of its budget matches us there, modular recombination is not earning its keep and we would say so.

Why not just use the low-level value function as the feasibility check, as DIPPER does? This is the main threat, because a value function trained under physics randomization already estimates success from a start state. The argument for a density is that value networks extrapolate optimistically on states they never visited. A flow assigns those states low likelihood, and a flow separates "this physics is unfavorable" from "this state is unfamiliar" in a way a marginal value cannot. Both claims are testable. We measure handoff-failure prediction (area under the ROC curve) for each scorer on real handoffs. We also run the planner comparison at equal rollout budget. If the robust value function matches the density, we drop the flow and keep only the coverage-driven training of start states, which would then be the real contribution.

Why not just randomize every skill's start states very widely? Wide start-state randomization is the obvious cheap fix. Its failure mode is the over-conservatism already seen for physics: broad randomization gave 60% against 90% for learned randomization on real gear insertion in GoFlow. Dexterous skills trained from implausible grasps may also learn timid, slow strategies. We include a wide uniform start-state baseline with the same training budget and compare chain success and execution time. If it matches us, the learned coverage is unnecessary.

Proposed Experiments

In simulation we build multi-stage chains on Bi-DexHands' two 24-DoF Shadow Hands, composing grasp, handover, in-hand reorientation, and placement. We train the library on a subset of chains and test on held-out orderings. Physics is perturbed beyond the training distribution to mimic a reality gap. Metrics are full-chain success, per-boundary handoff failure rate, failure location (inside a skill versus at a handoff), and simulator steps spent on each new chain. No reported result was found on Lacuna for multi-stage chained Bi-DexHands tasks; the closest reference point is the multi-task collapse described above. We also run the planner on a single-hand library in OGBench-style long-horizon settings to check against Compositional Planning with Jumpy World Models, which reports 67–76% on Cube-4 with frozen policies and fixed switching probabilities. The comparison asks whether state-dependent density scoring helps there too.

On hardware we use a Kuka-Allegro arm-hand system comparable to ADEPT's. The test chain is grasp, then in-hand reorientation, then FMB asymmetric peg insertion, plus a held-out chain that inserts regrasp before placement on the dish rack. Every method gets the same trial count, objects, and initial-pose protocol. We compare against ADEPT's reported 8/10 and 6/10 and against our own rerun of it.

Three decisive checks determine whether the idea holds. First, the naive chain must fail mostly at handoffs. If failures sit inside skills, the premise is wrong and we redirect to skill robustness. Second, removing the coverage term must raise handoff failures on hardware, isolating the key ingredient. Third, the full method must beat the robust-value planner on held-out real chains, or the density's role is reduced to a training device. A positive result on all three would give a working recipe: dexterous skill libraries that are trained once in simulation and recombined on real hands without per-task training.