Humanoid Hanoi: Investigating Shared Whole-Body Control for Skill-Based Box Rearrangement

Minku Kim, Kuan-Chia Chen, Aayam Shrestha, Li Fuxin, Stefan Lee, Alan FernarXiv 2026 · arXiv

TL;DR

This paper introduces a modular control framework that sequences high-level loco-manipulation skills through a single, shared whole-body controller (WBC) to achieve long-horizon humanoid robot autonomy. To counter the distribution shifts that occur when chaining independently learned skills, the authors propose a rollout-based data aggregation technique that incrementally robustifies the shared WBC. The approach is validated on "Humanoid Hanoi," a newly introduced long-horizon box rearrangement benchmark, demonstrating successful multi-minute autonomous execution on the Digit V3 humanoid robot.

What Problem Does This Paper Address?

Executing long-horizon tasks with humanoid robots—such as rearranging boxes over several minutes—requires a seamless blend of locomotion and manipulation skills. While breaking these tasks down into modular, independently trained skills (like walking, picking, and placing) makes intuitive sense, naively chaining them together is notoriously brittle.

Two main issues arise:

  1. Low-Level Instability: Switching between skill-specific low-level controllers at skill boundaries alters closed-loop dynamics, often causing the robot to lose balance.
  2. Distribution Shift: Composing skills introduces novel robot states and high-level commands that the low-level controller never encountered during its isolated training phase. Small errors accumulate over time, pushing the robot into unrecoverable configurations.

The authors tackle these challenges by proposing an architecture that keeps the low-level whole-body controller fixed and shared across all skills, treating robustness as an incremental maintenance problem of this shared controller.

Methods

The control architecture is split into a hierarchical structure:

System Overview
  • High-Level Skills: A library of reusable skills (GoTo, Pickup, GoTo-with-box, and Place) implemented as LSTM policies trained via Reinforcement Learning (PPO). Instead of directly commanding joint torques, these skills output masked motion directives. This masking mechanism allows a skill to constrain only relevant parts of the body (e.g., arm positions for a pickup) while leaving other parts unconstrained.
  • Shared Whole-Body Controller (WBC): An always-on, task-agnostic Masked Humanoid Controller (MHC) takes the robot's proprioceptive state and the masked motion directives to output joint-level PD targets at 50 Hz. Because the WBC is shared, switching skills merely changes the high-level command stream rather than the underlying control laws.
  • WBC Coverage Expansion (Data Aggregation): To make the shared WBC robust to skill-induced distribution shifts, the authors implement a rollout-based data aggregation procedure. They run the high-level skills with the initial WBC in simulation, collect the closed-loop directive sequences from successful rollouts, add these sequences to the WBC's reference trajectory library, and continue training the WBC under domain randomization.

Datasets

  • AMASS Dataset: A large database of human motion capture data used for the initial pre-training of the Masked Humanoid Controller.
  • Humanoid Hanoi Benchmark: A novel, long-horizon box rearrangement benchmark introduced in this paper. Inspired by the classic Tower of Hanoi puzzle, it requires a humanoid robot to move three boxes of varying sizes and masses between three tower locations while respecting stacking constraints. A full solution requires sequencing 7 moves, totaling 28 individual skill executions over an extended duration.

Key Insights and Ideas

  • Robustness as WBC Maintenance: Instead of modifying low-level control on a per-skill basis (e.g., via residual policies or per-skill fine-tuning), the authors show that it is better to treat robustness as a data-coverage problem for a single, shared WBC.
  • The Power of Masked Directives: Utilizing a masked interface allows the WBC to autonomously complete unspecified degrees of freedom (like leg joints and torso pitch) in a dynamically consistent manner, ensuring the robot maintains balance regardless of the active manipulation skill.
  • Closed-Loop Data Aggregation Wins: Training the WBC on actual closed-loop trajectories generated by the high-level skills exposes the controller to realistic tracking errors and deployment distributions, which significantly eases sim-to-real transfer.

Key Results

The authors compared their Extended (data aggregation) approach against a Base frozen WBC, a Finetune baseline, and a Residual policy baseline.

Fig. 4: Cumulative task success rates for the Humanoid Hanoi benchmark. Each move consists of four skills: GoTo, Pickup, GoTo with Box, and Place, shown as small markers, with large markers indicating move completion. Drops between consecutive points indicate failures in the subsequent skill (e.g., a drop from Pickup to GoTo with Box indicates failures during the GoTo with Box execution).
  • Simulation Performance: In evaluations featuring heavy domain randomization (DR), the Extended approach achieved a 49% complete-task success rate on the Humanoid Hanoi benchmark, substantially outperforming Base (25%), Finetune (21%), and Residual (19%).
  • Hardware Deployment: The framework successfully transferred to the physical Digit V3 humanoid robot. Across three workspace configurations, Digit achieved an overall 40% full-task success rate (6 out of 15 trials), executing fully autonomous, continuous rearrangement sequences lasting over 5 minutes per successful trial.
Fig. 1: Humanoid Hanoi, a problem instance from the Tower- of-Hanoi box rearrangement benchmark. The robot moves boxes between three towers (T1–T3) while respecting stacking constraints. The panels illustrate representative stages of a successful hardware execution (5+ min), with the corresponding symbolic state shown on the right. This benchmark stresses long-horizon autonomy by requir- ing repeated skill chaining with precise placement under constraints.

Limitations and Future Work

Despite the promising results, the authors noted several failure modes that highlight areas for future research:

  • Perception Vulnerability: The real-world system relied on AprilTags for localization and box tracking. Occlusions or lighting changes frequently caused tag loss, leading to failed grasps or placements.
  • Object-Unaware Locomotion: The GoTo locomotion skills were blind to the environment's obstacles. After placing a box, the robot would occasionally collide with the newly formed stack when walking backward or turning in place.
  • Accumulated Placement Errors: Small errors in base positioning before entering the Place skill directly translated into box alignment offsets, occasionally causing stacks to destabilize and collapse in later stages of the puzzle.

Summary updated

Abstract

We investigate a skill-based framework for humanoid box rearrangement that enables long-horizon execution by sequencing reusable skills at the task level. In our architecture, all skills execute through a shared, task-agnostic whole-body controller (WBC), providing a consistent closed-loop interface for skill composition, in contrast to non-shared designs that use separate low-level controllers per skill. We find that naively reusing the same pretrained WBC can reduce robustness over long horizons, as new skills and their compositions induce shifted state and command distributions. We address this with a simple data aggregation procedure that augments shared-WBC training with rollouts from closed-loop skill execution under domain randomization. To evaluate the approach, we introduce Humanoid Hanoi, a long-horizon Tower-of-Hanoi box rearrangement benchmark, and report results in simulation and on the Digit V3 humanoid robot, demonstrating fully autonomous rearrangement over extended horizons and quantifying the benefits of the shared-WBC approach over non-shared baselines. Project page: https://osudrl.github.io/Humanoid_Hanoi/
BibTeX
@misc{kim2026humanoid,
  author = {Minku Kim and Kuan-Chia Chen and Aayam Shrestha and Li Fuxin and Stefan Lee and Alan Fern},
  title = {{Humanoid Hanoi: Investigating Shared Whole-Body Control for Skill-Based Box Rearrangement}},
  year = {2026},
  eprint = {2602.13850},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.13850},
}
Figure