Abstract
Humanoid robots have achieved significant milestones in locomotion, yet they often lack the fine-grained stabilization required for delicate manipulation tasks during movement. This paper introduces SoFTA (Slow-Fast Two-Agent), a reinforcement learning framework that decouples control into a high-frequency upper-body agent for stabilization and a lower-frequency lower-body agent for robust locomotion. By addressing the temporal and objective mismatches between these tasks, SoFTA achieves a 2–5× reduction in end-effector acceleration compared to standard whole-body control, enabling humanoid robots to carry liquids with human-like stability.
Introduction
While modern humanoid robots can perform impressive feats like backflips or navigating rocky terrain, they remain surprisingly "clumsy" when it comes to fine-grained stabilization. A state-of-the-art humanoid tasked with carrying a full cup of coffee will typically spill the liquid due to the vibrations and oscillations inherent in bipedal walking. This research identifies a fundamental mismatch in task characteristics: locomotion is inherently "slow" and discrete, relying on noisy foot contacts, whereas end-effector (EE) stabilization requires "fast," high-precision corrections to react to body vibrations.
The SoFTA Framework
The core innovation of this work is the Slow-Fast Two-Agent (SoFTA) framework. Instead of a single reinforcement learning (RL) agent controlling the entire body, the system is split into two specialized components to handle the conflicting demands of movement and stability.
Temporal and Objective Decoupling
SoFTA employs two distinct strategies to achieve stability:
- Temporal Decoupling: The upper-body agent (14 joints) operates at 100 Hz, allowing for rapid disturbance rejection. The lower-body agent (13 joints) operates at 50 Hz, prioritizing gait robustness. High-frequency leg control is often counterproductive due to sensor noise, but high-frequency arm control is essential to cancel out vibrations from foot impacts.
- Objective Decoupling: Separate reward groups are used for each agent. The upper body is rewarded for minimizing linear/angular acceleration and gravity tilt of the end-effector, while the lower body focuses on velocity tracking and gait symmetry.
Both agents share the same full-body proprioceptive state, ensuring coordinated movement despite their independent action frequencies.
Experimental Results
Training was conducted in the Isaac Gym simulation environment at 200 Hz. To ensure the policy learned reactive compensation rather than just static poses, the robot was given randomized end-effector target commands during training. This encouraged the emergence of active compensation behaviors, such as the wrist "rolling" to counteract base movement.
Real-World Validation
The system was deployed on the Unitree G1 and Booster T1 humanoid robots. SoFTA reduced end-effector acceleration by 50–80% compared to standard whole-body RL and the robots' default controllers. During diverse locomotion, the system maintained end-effector acceleration below 2 m/s², approaching human-level stability.
Conclusion
SoFTA demonstrates that decoupling control frequencies and objectives can solve the "clumsiness" problem in humanoid robots. By allowing the upper body to act as a high-speed stabilizer for the lower body's robust locomotion, robots can finally perform delicate tasks in dynamic environments. Future work will explore more fluid boundaries between locomotion and manipulation for complex tasks like heavy lifting.
Summary updated