Introduction
Simulating realistic human motion in physics-based environments remains a significant challenge in computer graphics and robotics. While kinematic methods can produce visually appealing results, they often suffer from artifacts like "foot sliding" or "ghosting." Physics-based reinforcement learning (RL) offers a more grounded alternative, yet these models frequently struggle to replicate highly agile human behaviors—such as ballet pirouettes or acrobatic flips—due to "dynamics mismatch." This mismatch occurs because simplified simulated humanoids lack the biomechanical complexity of real humans, making certain captured motions physically impossible to execute within the simulator's constraints.
To address this, we introduce Residual Force Control (RFC), a technique that augments a humanoid's internal joint torques with external "residual forces." This approach enables virtual characters to imitate complex, high-agility motions that were previously unattainable for standard physics-based controllers.
Methodology
The core innovation of this work is the integration of external residual forces into the agent's action space. These forces act as "invisible helping hands" that provide the necessary pushes or pulls to compensate for simulation inaccuracies.
Residual Force Control (RFC)
RFC allows the RL agent to apply external forces to maintain balance and accuracy. To ensure the resulting motion remains physically plausible, a heavy penalty is applied to the use of these forces, forcing the agent to rely on internal joint torques whenever possible. The paper explores two variants:
- RFC-Explicit: Models forces applied to specific body parts, such as the feet.
- RFC-Implicit: A more computationally efficient version that applies a general torque to the character's root.
Dual-Policy Framework
For long-term and diverse motion synthesis, a two-stage pipeline is employed:
- Kinematic Policy: A Conditional Variational Autoencoder (CVAE) generates physically plausible future poses based on historical data.
- RFC Policy: A low-level controller executes these predicted poses within the physics engine, utilizing residual forces to bridge the gap between kinematic "dreams" and rigid-body physics.
Results and Evaluation
The framework was evaluated across three major benchmarks: CMU MoCap (agile motions), Human3.6M (long-term synthesis), and EgoMocap (head-mounted camera data).
As shown in Figure 2, RFC-based methods demonstrate superior learning efficiency and higher final performance compared to state-of-the-art baselines like DeepMimic. The addition of residual forces provides a smoother gradient for the RL agent, leading to faster convergence even in complex tasks like ballet and acrobatics.
Quantitatively, the RFC-based dual policy achieved significantly lower Mean Angle Error (MAE) and Final Angle Error (FAE) on the Human3.6M dataset compared to prior physics-based and kinematic methods. Furthermore, the RFC-Implicit model is highly efficient, adding only ~0.4ms of computational overhead per step, making it suitable for real-time applications.
Discussion and Future Work
RFC serves as a vital "invisible bridge" between kinematic data and rigid-body physics. By acknowledging the inherent imperfections of simulation models, it allows for much higher visual fidelity in motion imitation. Moreover, the framework enables task-free synthesis, allowing characters to generate "infinite-horizon" natural behaviors without explicit goal guidance.
While RFC cannot be directly deployed on physical hardware due to its reliance on external forces, it offers a powerful tool for simulation-based training. Future work may involve using RFC as a "warm-up" phase, gradually decaying the residual forces to zero to facilitate real-world transfer. Additionally, the magnitude and location of the required residual forces could inform the design of more capable physical robots by highlighting areas where better weight distribution or higher torque is needed.
Summary updated