Letting the Robot Ask for Help: Learner-Allocated Assistance for Cross-Hand Dexterous Retargeting

Teaching a robot hand to copy a human doing a two-handed task, such as opening a waffle iron, fails for a basic reason. The robot's fingers have different lengths, joints and strength, so the human's motions cannot be replayed directly. The strongest current method, DexMachina, sidesteps this with "virtual object controllers." These are invisible simulated forces that at first move the object along the human's path for the robot, then fade out on a fixed timetable until the robot does all the work. The timetable is the same for every robot hand and every moment of the task. Yet the difficulty of copying a human is concentrated in particular moments, like a regrasp, a lid lift or a handoff, and those moments differ from hand to hand. We propose to let the learner decide where the help goes. The policy outputs, alongside its motor commands, how much virtual-controller support it wants at each step. A shrinking overall help budget forces it to spend that support only where its own body cannot yet cope.

The idea comes from work on learner agency in people, where letting a learner choose what to review instead of following a fixed sequence improves mastery at the same time cost (Teaching Active Human Learners). That literature also gives a warning: learners who hand off the hard part do worse once the help is removed (How AI Impacts Skill Formation). We expect two results. First, when the total help budget is held fixed, letting the learner allocate it should beat a fixed decay, by the largest margin on hands whose shape differs most from a human hand. Second, the record of where each hand asked for help should be a readable map of where the embodiment gap actually bites for that hand.

Core Claim

At equal total assistance, a policy that chooses when and how strongly to invoke virtual object controllers should reach higher success without assistance than DexMachina's uniform decay schedule. It should also need fewer simulation steps. The reason is that embodiment mismatch produces failures at particular states, and a schedule indexed only by training time cannot target them. A schedule indexed by demonstration segment can partly target them, so it is the key competitor and we test against it directly. A secondary claim is that these help requests line up with independent measures of kinematic infeasibility for each hand, which would make them a diagnostic for hardware choice.

Positioning Against Prior Work

The work that motivates this proposal comes from two directions. On the embodiment side, DexMV showed that direct fingertip mapping makes intermediate robot finger links pass through objects, which it calls "ghost fingers." It also showed that pure reinforcement learning on multi-fingered relocation reaches 0% success. DexMachina showed that kinematic retargeting and task-reward-only learning both fail on long bimanual articulated tasks built from ARCTIC, and fixed this with decaying virtual controllers. Its curriculum is externally scheduled, and this is the one thing we change. We keep its task rewards, hands and objects. On the agency side, ALTA (Teaching Active Human Learners) lets a human learner choose between a new example and reviewing an old one. It reports 70.18% accuracy on Butterfly versus 64.82% for a fixed curriculum at nearly identical time. Its setting is binary classification, and we carry the "learner chooses new versus review" structure into continuous control.

Three works connect agency and embodiment differently, and each one constrains the design. ASH has an agent detect that it is stuck and retrieve internet video, but only for discrete-action 2D games. EgoAVFlow gives the robot control over its camera viewpoint, but the arm itself only executes its plan. CrossDex lets a student query privileged teachers through DAgger (dataset aggregation, in which the student acts and a teacher labels the states it visits). That requires a per-hand expert to already exist, and producing such an expert is the problem we are solving. None of these lets a learner allocate physical assistance within an episode.

The skill-formation result (How AI Impacts Skill Formation) plays a constraining role. In that study, novices who delegated code generation scored 24–39% on comprehension, compared with 65–86% for those who interrogated the code. A robot given cheap help could likewise learn to depend on it, so every headline number in this proposal is measured with help switched off, and under perturbation.

Research Program

The method keeps DexMachina's simulation setup: floating-wrist bimanual hands, articulated objects with privileged state, and the object-trajectory tracking reward. We add a scalar help channel $h_t \in [0,1]$ to the policy's output, which scales the gains of the virtual object controllers at step $t$. We also give the learner a start-segment choice, analogous to ALTA. At each reset, a small learned selector chooses whether to begin from the next unmastered segment of the demonstration or to review an earlier one. Its scoring uses ALTA-like terms for recency of failure and repetition. Each episode starts from the object state at the beginning of the chosen segment, and the hand starts from the robot's own previously reached configuration there, not from a retargeted human pose. Training solves

$$\max_\pi \; \mathbb{E}\Big[\sum_t r_t\Big] \quad \text{s.t.} \quad \mathbb{E}\big[\bar h\big] \le B_k,$$

where $r_t$ is DexMachina's task reward, $\bar h$ is the mean help level over an episode, and $B_k$ is the allowed help budget at training iteration $k$. The constraint is enforced with a Lagrange multiplier updated by dual ascent.

The budget $B_k$ is set to exactly the mean controller strength DexMachina's decay schedule would apply at iteration $k$. Both methods therefore consume identical total assistance over training, and only its allocation differs. A fixed fraction of training episodes, 25%, runs with help forced to zero, so that the policy cannot become conditioned on help always being available.

The program has three parts. The first is the method. The second is the matched-budget comparison across all six DexMachina hands and five tasks. The third is a diagnostic study. For each hand, we compute the per-segment average of $h_t$ and correlate it with two independent difficulty signals. One is the residual of a DexMV-style task-space retargeting of the human motion to that hand. The other is the per-segment failure rate of the fixed-schedule baseline.

Baselines

The naive reference points are task-reward-only reinforcement learning and kinematic retargeting followed by joint tracking. Both are rerun on the DexMachina tasks with the same PPO (proximal policy optimization) implementation and simulation budget. DexMachina reports that both fail on long-horizon tasks such as Waffleiron-300, so they bound the low end.

The strongest existing method is DexMachina itself, with its uniform controller decay (DexMachina). The file reports only qualitative superiority over its baselines, with no reported result found on Lacuna for the exact success percentage on Waffleiron-300 or for benchmark-wide numbers. We will therefore rerun it from its released setup with identical seeds (five per hand–task pair), environment steps and network size, and report our reproduction as the reference. To be fair to it, we also give DexMachina a per-hand tuned version. We search its decay rate and shape per hand with the same number of tuning trials we spend on our Lagrangian learning rate and help-free fraction.

The most dangerous competitor is per-segment adaptive decay. This is the simple heuristic in which each demonstration segment's controller strength decays only once help-free success on that segment passes a threshold, under the same total budget. It captures "help where needed" at segment granularity without any learned help action. We also run a learning-progress segment sampler that chooses start segments without a help channel, which tests review agency alone.

The ablations isolate the two ingredients: help channel without segment choice, segment choice without help channel, and help channel with no budget. The last one probes delegation. Beating the field means two things. Help-free success must be higher than tuned DexMachina and per-segment decay at matched budget, with non-overlapping 95% bootstrap intervals across seeds, on at least four of six hands. And the advantage should be largest for the hands DexMachina found hardest, the smaller and under-actuated ones.

Why not just…?

Why not just train from scratch with reinforcement learning and domain randomization? The evidence in the file is direct. DexMV reports 0% success for pure reinforcement learning on multi-fingered relocation, and DexMachina reports task-reward-only learning stalling early on bimanual articulated tasks. We still include it as the floor.

Why not just tune DexMachina's decay schedule per hand? This is a real alternative, and the tuned-schedule baseline exists to test it. A per-hand global schedule can adapt how fast help fades, but not where it is spent within a task. If the tuned schedule matches our method, the conclusion would be that hand-level adaptation suffices and that within-task allocation adds nothing. We would report that outcome.

Why not just decay help per segment based on success? This is the most serious threat. If failures are tied to segments, this heuristic may capture most of the gain. Our claim is that embodiment failures are tied to states, such as a particular contact configuration that can occur at different times across rollouts, rather than to fixed time windows. The matched comparison decides it. We also report results as a function of segment length. If the advantage disappears as segments become short, the heuristic is the better tool and the learned help channel is unnecessary.

Why not just reset to intermediate demonstration states, a reverse curriculum, with no virtual controllers? Starting mid-demonstration requires a valid robot hand configuration holding the object at that moment. The only source for one is retargeting the human hand, and DexMV and DexMachina show that retargeting is unreliable precisely in contact-rich moments. Our segment selector avoids this by resetting the hand to configurations the robot itself has reached. Our segment-choice-only ablation approximates the reverse-curriculum idea under that constraint.

Why not just use a DAgger teacher as in CrossDex? DAgger needs a per-hand expert, and on these long articulated tasks no such expert exists without first solving the problem.

Proposed Experiments and Decisive Checks

The primary experiment covers all six hands and five tasks of the DexMachina benchmark, with five seeds each. We measure help-free task success, object pose and articulation tracking error, and the environment steps needed to reach 50% help-free success.

The first decisive check is the matched-budget comparison. If learner allocation does not beat both tuned DexMachina and per-segment decay at equal total assistance, the core claim fails.

The second check targets delegation. Policies are evaluated with object mass and friction randomized ±30% and with random external pushes, again with help off. The prediction, mirroring How AI Impacts Skill Formation, is that the unbudgeted-help ablation shows the largest drop from nominal to perturbed success. If the budgeted learner were also brittle, the budget would not be preventing dependence.

The third check is the diagnostic. Per-segment help maps should correlate positively, by Spearman correlation, with per-hand retargeting residuals and with baseline failure rates. Across hands, total requested help should be ordered consistently with DexMachina's observation that larger, fully actuated hands learn more easily. A null correlation would mean the help channel is being used for reasons unrelated to the embodiment gap. That would undercut the interpretation even if success improved.

As an optional extension, the rigid-object pouring task from Human2Sim2Robot can test whether learner-allocated object guidance also helps the single-demonstration regime. That paper reports outperforming open-loop replay by 55% and diffusion behavior cloning by over 68%.