Learning Getting-Up Policies for Real-World Humanoid Robots

Xialin He, Runpei Dong, Zixuan Chen, Saurabh Gupta

TL;DR

HUMANUP is a novel two-stage reinforcement learning framework that enables human-sized humanoid robots to autonomously recover from falls across diverse, challenging real-world terrains. By decoupling trajectory discovery from hardware deployment constraints, the system successfully transfers learned getting-up and rolling-over policies to the Unitree G1 robot, significantly outperforming hand-crafted commercial controllers.

What Problem Does This Paper Address?

While reinforcement learning (RL) has revolutionized humanoid locomotion (like walking and running), automatic fall recovery remains a major bottleneck for deploying these robots in the real world. If a humanoid falls, it can land in an unpredictable configuration on highly irregular terrain.

Standard RL locomotion tricks fail here for three reasons:

  1. Non-periodic behavior: Locomotion relies on cyclic, predictable left-right stepping patterns. Getting up requires figuring out an entirely unstructured sequence of contacts.
  2. Richness in contact: Unlike walking, which only involves foot-ground contact, getting up requires the robot to use its entire body (arms, torso, knees) to interact with the environment.
  3. Reward sparsity: In locomotion, velocity tracking provides immediate, dense feedback. When getting up, a robot often has to make "negative progress" first—such as tilting its torso down for several seconds—before it can successfully hoist itself up, making standard reward structures ineffective.

Methods

The authors introduce HUMANUP, a two-stage training framework that establishes a clever curriculum to bridge the sim-to-real gap:

HUMANUP System Overview
  • Stage I: Discovery Policy: This stage focuses purely on solving the task in an idealized environment. The robot learns a getting-up trajectory using a simplified collision mesh, a canonical starting pose, and very weak control regularization. This allows the RL agent to freely explore and discover successful motions without being penalized for jerky movements or high torques.
  • Stage II: Deployable Policy: The discovered trajectory is slowed down (by 8×) and used as a reference target. In this stage, the robot learns to track this motion under strict real-world constraints: a full whole-body collision mesh, heavy control regularization (penalizing joint jitter and excessive torque), posture randomization, and domain/terrain randomization.

Datasets

To train and evaluate the system's robustness, the authors curated a simulated dataset $\mathcal{P}$ consisting of:

  • 20,000 supine poses (lying face up)
  • 20,000 prone poses (lying face down)

These configurations were generated in Isaac Gym by randomizing the robot's initial joint angles, dropping it from a height of 0.5 meters, and letting it settle for 10 seconds to resolve self-collisions. 10,000 poses from each set were held out exclusively for evaluation.

Key Insights and Ideas

  • The "Hard-to-Easy" and "Easy-to-Hard" Curriculum: Splitting the pipeline into two stages creates a dual curriculum. Moving from Stage I to Stage II represents a hard-to-easy transition in terms of task difficulty (shifting from open-ended task discovery to simple motion tracking), but an easy-to-hard transition for regularization and environmental variability. Trying to learn both simultaneously fails completely.
  • Soft Symmetry Rewards: Instead of forcing strict mathematical bilateral symmetry (which limits the robot's degrees of freedom and generalization), HUMANUP uses a soft symmetry reward. This guides the agent toward balanced motions while allowing it to break symmetry when dealing with uneven terrains.
  • Leveraging Full URDF Geometry: Training Stage II with the official, high-fidelity whole-body URDF mesh—rather than a simplified capsule approximation—proved essential for real-world transfer, as it accurately captured the complex contact dynamics of the robot's shell against the ground.

Key Results

The learned policies were deployed directly on a real-world Unitree G1 humanoid robot across six diverse terrains: concrete, brick, stone tiles, muddy grass, grass slopes (~10°), and a snowfield.

Real-World Results across Terrains
  • High Success Rates: In simulation, HUMANUP achieved a 95.34% success rate from supine poses and 92.10% from prone poses. In the real world, it achieved an overall 78.3% getting-up success rate from supine positions and 98.3% for rolling over.
  • Outperforming the Manufacturer Baseline: The default, hand-crafted controller provided by Unitree only succeeded on flat, firm concrete. HUMANUP successfully navigated bumps, slopes, and slippery snow.
  • Efficiency and Safety: HUMANUP stands up in about 6 seconds—over 2× faster than the manufacturer's controller (11 seconds). Furthermore, by utilizing the robot's stronger leg motors rather than forcing it to push up heavily with its arms, HUMANUP kept motor temperatures significantly lower, preventing hardware damage.
Execution and Motor Temperature Comparison

Limitations and Future Work

Despite its success, the authors note several limitations:

  1. Unnatural Behaviors: Because the reward function is under-specified, the robot occasionally exhibits "reward hacking" behaviors, such as raising an arm awkwardly in the air for balance.
  2. Simulation Dependency: The framework relies heavily on high-frequency (1 kHz) physics simulation in Isaac Gym; modeling even more complex interactions (like deformable mud or deep snow) remains a challenge.
  3. Complex Terrains: The current policy has not yet been extended to structured, highly uneven environments like stairs or steep, jagged obstacles.

Summary updated

Abstract

Automatic fall recovery is a crucial prerequisite before humanoid robots can be reliably deployed. Hand-designing controllers for getting up is difficult because of the varied configurations a humanoid can end up in after a fall and the challenging terrains humanoid robots are expected to operate on. This paper develops a learning framework to produce controllers that enable humanoid robots to get up from varying configurations on varying terrains. Unlike previous successful applications of learning to humanoid locomotion, the getting-up task involves complex contact patterns (which necessitates accurately modeling of the collision geometry) and sparser rewards. We address these challenges through a two-phase approach that induces a curriculum. The first stage focuses on discovering a good getting-up trajectory under minimal constraints on smoothness or speed / torque limits. The second stage then refines the discovered motions into deployable (i.e. smooth and slow) motions that are robust to variations in initial configuration and terrains. We find these innovations enable a real-world G1 humanoid robot to get up from two main situations that we considered: a) lying face up and b) lying face down, both tested on flat, deformable, slippery surfaces and slopes (e.g., sloppy grass and snowfield). This is one of the first successful demonstrations of learned getting-up policies for human-sized humanoid robots in the real world.
BibTeX
@misc{he2025learning,
  author = {Xialin He and Runpei Dong and Zixuan Chen and Saurabh Gupta},
  title = {{Learning Getting-Up Policies for Real-World Humanoid Robots}},
  year = {2025},
  eprint = {2502.12152},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2502.12152},
}
Figure