Abstract
Physically simulated humanoids offer a path toward realistic and interactive virtual characters, yet they are notoriously difficult to control. We present Perpetual Humanoid Control (PHC), a system that enables avatars to mimic complex human motions in real-time without requiring resets. By employing a Progressive Multiplicative Control Policy (PMCP), the system scales to large-scale motion datasets and naturally recovers from falls or noisy input. PHC eliminates the need for external "Residual Force Control," resulting in physically plausible movements that are robust enough for live applications.
Introduction
The control of physically simulated humanoids is a long-standing challenge in computer animation and robotics. Unlike kinematic animation, simulated characters must contend with gravity, momentum, and balance. Traditional approaches often fail when faced with three primary issues: the Reset Problem, where characters must teleport back to a start pose after a fall; the "Hand of God", or the use of unphysical external forces (Residual Force Control) to maintain balance; and Catastrophic Forgetting, where a model loses proficiency in simple tasks while learning complex ones.
Methods
The core of PHC is the Progressive Multiplicative Control Policy (PMCP). This architecture adopts a "divide and conquer" strategy to manage the complexity of human motion.
Progressive Learning and Composition
Instead of a single monolithic network, PMCP utilizes a series of "primitives." The system begins with a small network trained on simple motions. As it encounters more difficult sequences, it freezes the existing weights and adds a new primitive to handle the failure cases. A high-level Composer network then learns to blend these primitives in real-time, ensuring smooth transitions between diverse actions.
Stability and Recovery
To ensure "perpetual" operation, the authors introduced two key mechanisms:
- Fail-state Recovery: A dedicated primitive is trained specifically for the "getting up" motion. If the character falls or deviates significantly from the reference, this module restores the character to its feet.
- Relaxed Early Termination (RET): During training, standard methods stop the simulation if any joint deviates from the reference. RET allows for slight deviations in the extremities (ankles and toes) to prioritize overall balance, significantly enhancing stability.
Key Insights
A significant finding of this work is that 3D keypoints are sufficient for high-fidelity imitation, removing the need for complex joint rotation data. This allows the system to be driven by standard webcam-based pose estimators. Furthermore, the physics simulation acts as a natural regularizer, cleaning up the jitter often found in AI-generated pose data.
Results
PHC demonstrates state-of-the-art performance across several benchmarks. It successfully imitated 98.9% of the AMASS dataset—comprising over 40 hours of motion—without any external stabilizing forces.
In evaluations using the Human3.6M dataset, PHC maintained a ~90% success rate on noisy, real-world video data. In contrast, previous methods often saw success rates drop below 20% when their external forces were disabled. The system is highly efficient, running at over 30 FPS on consumer-grade GPUs, and the recovery module provides nearly 100% reliability in returning a fallen character to a tracking state within seconds.
Conclusion and Future Work
Perpetual Humanoid Control provides a robust framework for real-time, physics-based animation. While the system currently struggles with extreme dynamic motions like backflips and lacks environmental awareness (e.g., navigating terrain or interacting with objects), it sets a new standard for stability and scale. Future research will focus on integrating terrain perception and closing the loop between vision and physics to further improve naturalness and responsiveness.
Summary updated