Why Koopman Structure Does Not Help Differentiable World Models for Humanoid Whole-Body Tracking

Haizhou Ge, Mengshuo Wang, Yufei Jia, Lei Han, Lu Shi, Guyue Zhou, Ruqi Huang

TL;DR

While Koopman theory promises to simplify complex nonlinear dynamics by mapping them into a linear "lifted" space, this paper demonstrates that it actually hurts performance when training humanoid tracking policies. The authors find that the structural constraints of Koopman models—specifically their "action-affine" nature—introduce significant prediction errors because the model struggles to infer the policy's intended actions from state information alone.

What Problem Does This Paper Address?

In the world of humanoid robotics, "differentiable world models" are a powerful tool. They allow researchers to train control policies by backpropagating gradients directly through a learned simulation of the robot's physics. Traditionally, these world models are black-box Multi-Layer Perceptrons (MLPs).

However, researchers have long been tempted by Koopman operator theory. The idea is elegant: even if a system (like a humanoid robot) is wildly nonlinear, there exists a high-dimensional "lifted" space where its dynamics evolve linearly. If we could use a Koopman model as our world model, we could theoretically leverage powerful tools from linear control theory.

The authors of this paper set out to answer a simple but critical question: Does this theoretical elegance actually lead to better humanoid tracking in practice?

Methods

The researchers integrated a deep bilinear Koopman world model into the SuperMimic pipeline, a state-of-the-art framework for humanoid whole-body tracking. They compared this against the standard MLP-based world model using a simulated Unitree G1 humanoid.

Key technical aspects included:

  • Action-Affine Structure: Because the Koopman model evolves linearly in the lifted space, the resulting prediction in physical space becomes "action-affine" ($\Delta = a(x) + M(x)u$). This means the change in state is a linear function of the action $u$, with a state-dependent offset $a(x)$ and slope $M(x)$.
  • Re-Lifting: The authors found that "lifting once" and rolling out in the latent space caused the model to diverge. To make it work, they had to "re-lift" the state at every single step, effectively re-encoding the physical state back into the Koopman space.
  • Controlled Comparison: They kept the rest of the training pipeline identical, changing only the world model architecture to isolate the effects of the Koopman structure.

Datasets

  • LAFAN1: A library of 40 human motion clips (approx. 2.45 hours) used for the primary humanoid tracking task.
  • Gymnasium MuJoCo: To ensure their findings weren't specific to humanoids, they also tested on standard benchmarks including Ant-v5, Humanoid-v4, Walker2d-v4, HalfCheetah-v4, and Hopper-v4.

Key Insights and Ideas

  • The "Operating Point" Problem: This is the paper's most significant finding. A policy usually acts near a specific "operating point" (its noise-free action). An action-affine model (like Koopman) must essentially "guess" this operating point from the state alone to be accurate. If the state doesn't contain enough information to predict what the policy intends to do, the model's prediction fails.
  • Structural Cost over Curvature: One might assume Koopman models fail because physics is too "curved" (nonlinear) for an affine approximation. Surprisingly, the authors show that curvature only explains a small fraction of the error. The majority of the "excess error" comes from the model's inability to learn the correct intercept at the operating point.
  • Re-Lifting Negates the Benefit: By re-lifting at every step to maintain stability, the model never actually uses the "linear evolution" that makes Koopman theory attractive. It just becomes a restricted, less accurate version of an MLP.

Key Results

The quantitative results were overwhelmingly in favor of the standard MLP:

  • Performance: The MLP achieved a return of 0.830, while the Koopman model reached only 0.823.
  • Accuracy: On identical offline data, the Koopman model's one-step prediction error was 1.3 to 1.6 times higher than the MLP's.
  • Efficiency: Training with the Koopman structure was roughly 15% slower and required more parameters (13.4M vs 8.8M).

As seen in Figure 1, the Koopman model (blue) consistently shows higher 8-step error throughout the training process compared to the MLP (black).

Fig. 1: Online G1 training (mean and range over seeds); (b) is a moving average over 500 iterations.

Figure 2 further illustrates that this accuracy gap persists across almost all MuJoCo tasks, with the Koopman model (blue squares) consistently performing worse than the MLP baseline (the 1.0 line).

Fig. 2: Offline one-step test error relative to the MLP world model (marker: mean of the 3 per-seed ratios; whiskers: range). G1: a fresh collection of the tracking data; others: Gymnasium MuJoCo datasets. Given µ: all world models, the MLP included, also read µ; n/m: not measured.

Limitations and Future Work

The authors acknowledge that their conclusions are specific to the "fixed lift" Koopman structure used in differentiable world models for high-dimensional tracking.

  • Scope: The findings might not apply to models where the action enters through a nonlinear encoder or where the observables depend directly on the input.
  • Potential Niche: Koopman structure might still be beneficial in scenarios where its linearity is strictly required, such as convex control of low-dimensional subsystems or very long-horizon open-loop planning.
  • Future Direction: A key open question is how to design world models that can capture the "operating point" information more effectively without sacrificing the benefits of structured dynamics.

Summary updated

Abstract

We test whether Koopman structure, linear dynamics in a lifted state space, helps when a humanoid whole-body tracking policy is trained by backpropagating through a differentiable world model, replacing the MLP world model of an otherwise unchanged pipeline for simulated Unitree G1 tracking. It does not: return is at best equal (0.823 versus 0.830), one-step error on identical data is 1.3 to 1.6 times the MLP's, and training is about 15% slower. Lifted rollouts diverge or fail to train a policy; of the schemes tested, re-lifting at every imagined step trains best. Re-lifting never uses the lifted linear dynamics and leaves a model affine in the action, with state-dependent offset and slope; an action-affine MLP also predicts worse, so the cost belongs to this class. At a fixed state, the curvature of the true dynamics in the action explains only part of it. Across states, the policy's actions span a curved region, so an affine model fits policy data only as the tangent at the policy's action, the operating point: on G1, the MLP's tangent there is as accurate as the MLP; at the mean action, 31 times worse. The offset must encode, from the state alone, where the policy acts, whereas an MLP reads the action. Supplying the policy's action removes most of the remaining gap on G1, Ant and Humanoid. The only online repair approaching the MLP feeds the action into the nonlinear lift, giving up the structure. The MLP world model thus remains the better choice here.
BibTeX
@misc{ge2026why,
  author = {Haizhou Ge and Mengshuo Wang and Yufei Jia and Lei Han and Lu Shi and Guyue Zhou and Ruqi Huang},
  title = {{Why Koopman Structure Does Not Help Differentiable World Models for Humanoid Whole-Body Tracking}},
  year = {2026},
  eprint = {2611.00951},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00951},
}
Figure