TL;DR
While Koopman theory promises to simplify complex nonlinear dynamics by mapping them into a linear "lifted" space, this paper demonstrates that it actually hurts performance when training humanoid tracking policies. The authors find that the structural constraints of Koopman models—specifically their "action-affine" nature—introduce significant prediction errors because the model struggles to infer the policy's intended actions from state information alone.
What Problem Does This Paper Address?
In the world of humanoid robotics, "differentiable world models" are a powerful tool. They allow researchers to train control policies by backpropagating gradients directly through a learned simulation of the robot's physics. Traditionally, these world models are black-box Multi-Layer Perceptrons (MLPs).
However, researchers have long been tempted by Koopman operator theory. The idea is elegant: even if a system (like a humanoid robot) is wildly nonlinear, there exists a high-dimensional "lifted" space where its dynamics evolve linearly. If we could use a Koopman model as our world model, we could theoretically leverage powerful tools from linear control theory.
The authors of this paper set out to answer a simple but critical question: Does this theoretical elegance actually lead to better humanoid tracking in practice?
Methods
The researchers integrated a deep bilinear Koopman world model into the SuperMimic pipeline, a state-of-the-art framework for humanoid whole-body tracking. They compared this against the standard MLP-based world model using a simulated Unitree G1 humanoid.
Key technical aspects included:
- Action-Affine Structure: Because the Koopman model evolves linearly in the lifted space, the resulting prediction in physical space becomes "action-affine" ($\Delta = a(x) + M(x)u$). This means the change in state is a linear function of the action $u$, with a state-dependent offset $a(x)$ and slope $M(x)$.
- Re-Lifting: The authors found that "lifting once" and rolling out in the latent space caused the model to diverge. To make it work, they had to "re-lift" the state at every single step, effectively re-encoding the physical state back into the Koopman space.
- Controlled Comparison: They kept the rest of the training pipeline identical, changing only the world model architecture to isolate the effects of the Koopman structure.
Datasets
- LAFAN1: A library of 40 human motion clips (approx. 2.45 hours) used for the primary humanoid tracking task.
- Gymnasium MuJoCo: To ensure their findings weren't specific to humanoids, they also tested on standard benchmarks including Ant-v5, Humanoid-v4, Walker2d-v4, HalfCheetah-v4, and Hopper-v4.
Key Insights and Ideas
- The "Operating Point" Problem: This is the paper's most significant finding. A policy usually acts near a specific "operating point" (its noise-free action). An action-affine model (like Koopman) must essentially "guess" this operating point from the state alone to be accurate. If the state doesn't contain enough information to predict what the policy intends to do, the model's prediction fails.
- Structural Cost over Curvature: One might assume Koopman models fail because physics is too "curved" (nonlinear) for an affine approximation. Surprisingly, the authors show that curvature only explains a small fraction of the error. The majority of the "excess error" comes from the model's inability to learn the correct intercept at the operating point.
- Re-Lifting Negates the Benefit: By re-lifting at every step to maintain stability, the model never actually uses the "linear evolution" that makes Koopman theory attractive. It just becomes a restricted, less accurate version of an MLP.
Key Results
The quantitative results were overwhelmingly in favor of the standard MLP:
- Performance: The MLP achieved a return of 0.830, while the Koopman model reached only 0.823.
- Accuracy: On identical offline data, the Koopman model's one-step prediction error was 1.3 to 1.6 times higher than the MLP's.
- Efficiency: Training with the Koopman structure was roughly 15% slower and required more parameters (13.4M vs 8.8M).
As seen in Figure 1, the Koopman model (blue) consistently shows higher 8-step error throughout the training process compared to the MLP (black).
Figure 2 further illustrates that this accuracy gap persists across almost all MuJoCo tasks, with the Koopman model (blue squares) consistently performing worse than the MLP baseline (the 1.0 line).
Limitations and Future Work
The authors acknowledge that their conclusions are specific to the "fixed lift" Koopman structure used in differentiable world models for high-dimensional tracking.
- Scope: The findings might not apply to models where the action enters through a nonlinear encoder or where the observables depend directly on the input.
- Potential Niche: Koopman structure might still be beneficial in scenarios where its linearity is strictly required, such as convex control of low-dimensional subsystems or very long-horizon open-loop planning.
- Future Direction: A key open question is how to design world models that can capture the "operating point" information more effectively without sacrificing the benefits of structured dynamics.
Summary updated