FORGE: Feasible Online Reduced-Order Reference Generation and Whole-Body Execution for Agile Humanoid Locomotion

Adrian B. Ghansah, Yujin An, Jeeseop Kim, Soon-Jo ChungarXiv 2026 · arXiv

TL;DR

FORGE introduces a two-stage reinforcement learning framework that enables humanoid robots to perform a wide array of agile maneuvers—from high-speed running to one-legged hopping—using a single unified policy. By coupling a kinematically-aware reduced-order model (ROM) for online reference generation with a full-order tracking controller, the system achieves impressive zero-shot sim-to-real transfer on the Unitree G1 humanoid.

What Problem Does This Paper Address?

Humanoid locomotion is notoriously difficult because these robots are high-dimensional, underactuated, and governed by nonlinear hybrid dynamics. Traditionally, researchers choose between two paths:

  1. Model-based Optimization: Using full-order models (FOM) to plan every joint movement is computationally expensive and often too slow for real-time adjustments.
  2. Reinforcement Learning (RL): While robust, training a single RL policy to handle a broad range of commands (like sprinting, turning, and hopping) without "mode-switching" logic is a significant challenge.

Furthermore, many RL methods rely on large datasets of human motion capture. However, human data doesn't always translate well to robot proportions, leading to "kinematically infeasible" references—motions the robot's joints simply cannot physically execute. FORGE aims to bridge this gap by generating high-quality, feasible references online without needing human data.

Methods

The authors propose a two-stage training architecture:

Stage 1: Reference Generation Instead of a simple point-mass model, the authors train a policy in a custom "ROM simulator." This simulator combines Single Rigid Body (SRB) dynamics (treating the robot as a lumped mass with fixed inertia) with the full humanoid kinematics. By including the robot's actual joint limits and geometry in this simplified stage, the policy learns to generate reference motions that are "kino-dynamically consistent"—meaning they respect both the laws of physics and the robot's physical structure.

Fig. 2: An overview of the two-stage training architecture. (Left) Illustrates the Stage 1 training, in which a command- conditioned whole-body reference generator is trained based on SRB dynamics and the full humanoid kinematics within a custom ROM simulator. (Right) Illustrates the Stage 2 training, in which a whole-body tracking policy for the full order system is trained to track the references generated in Stage 1.

Stage 2: Whole-Body Tracking The frozen policy from Stage 1 is used to generate references online. A second policy is then trained on the full-order dynamics (the "real" physics) to track these references. This stage focuses on closing the gap between the simplified ROM physics and the complex reality of limb inertia and contact forces.

Datasets

The framework does not rely on external datasets like human motion capture. Instead, it uses:

  • Custom ROM Simulator: A numerical environment for Stage 1 training that evaluates contact and collision detection.
  • MuJoCo: Used for visualization and Stage 2 training on the full-order model.
  • Unitree G1 Model: The specific kinematic and dynamic parameters of the Unitree G1 humanoid were used to ensure the generated motions were tailored to the hardware.

Key Insights and Ideas

  • Feasibility by Design: By incorporating full kinematics into the ROM training, the reference generator "knows" what the legs can and cannot do. This prevents the tracker from trying to follow impossible commands.
  • Unified Command Space: The framework doesn't use separate policies for walking vs. hopping. It treats hopping as just another command ($h_F$) that can be layered onto a velocity command.
  • Gait Phasing: The use of a monotonic phasing variable ($\phi$) allows the robot to seamlessly speed up or slow down its gait, providing a smooth transition between walking and running.
Fig. 3: (a) The different ROM foot contact modes. (b) The relationship between the swing foot phase and gait phase.

Key Results

The researchers demonstrated the system on a Unitree G1 humanoid with zero-shot transfer (no tuning on the physical robot):

  • Speed: Achieved longitudinal velocities from -2.0 m/s (backward) to 2.8 m/s (forward). In simulation, it reached up to 4.0 m/s.
  • Agility: Handled lateral velocities of $\pm 1.0$ m/s and yaw rates of $\pm 3.0$ rad/s.
  • Versatility: The robot successfully transitioned from walking to one-legged hopping on challenging terrain like sand.
  • Robustness: Even though trained only on flat ground, the robot could run on uneven, sloped grassy terrain.
Tracking performance showing the robot following time-varying velocity commands in simulation.

Limitations and Future Work

While the results are impressive, the authors noted a few constraints:

  • Hardware Limits: Some high-agility maneuvers (like 4.0 m/s sprinting or sustained hopping) were only demonstrated in simulation because they approached the maximum torque and power capabilities of the G1 hardware.
  • Terrain Awareness: The current policy was trained on flat ground. While it generalized well to grass and sand, explicit terrain-aware training could further improve robustness.
  • Future Directions: The team plans to conduct ablation studies to see which components (like the specific reward terms) contribute most to the performance and to compare FORGE against standard model-based controllers.

Summary updated

Abstract

This paper presents a two-stage learning framework for versatile and agile humanoid locomotion that couples online reduced-order model (ROM) reference generation with whole-body tracking control. In the first stage, we train a command-conditioned whole-body reference generator using single rigid body dynamics combined with full humanoid kinematics. In the second stage, the frozen ROM policy is rolled out online to generate references for training a whole-body tracking policy on the full-order dynamics model. Learning a kino-dynamically consistent reference generator in the simplified ROM space enables us to generate references over a broad command space and over multiple locomotion strategies using a single unified framework. We demonstrate the approach on a Unitree G1 humanoid robot, achieving longitudinal velocities from -2.0 m/s to 2.8 m/s, lateral velocities up to +-1.0 m/s, angular velocities up to +-3.0 rad/s, one-legged hopping, and continuous transitions across the commands, all using one unified reference policy and one whole-body tracking policy.
BibTeX
@misc{ghansah2026forge,
  author = {Adrian B. Ghansah and Yujin An and Jeeseop Kim and Soon-Jo Chung},
  title = {{FORGE: Feasible Online Reduced-Order Reference Generation and Whole-Body Execution for Agile Humanoid Locomotion}},
  year = {2026},
  eprint = {2610.37974},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.37974},
}
Figure