Auditing a Learned Simulator for Legged Robots: The Robotic World Model on ANYmal D

Joyjeet SingharXiv 2026 · arXiv

In the rapidly evolving field of robotics, "learned simulators"—neural networks that act as physics engines—are becoming a popular tool for training complex behaviors like walking. But how much can we trust these black-box models?

A new paper, "Auditing a Learned Simulator for Legged Robots," provides a rigorous, independent "stress test" of the Robotic World Model (RWM), a prominent system designed for the ANYmal D quadruped. Conducted entirely on a 2017 dual-core laptop without a GPU, this audit reveals critical gaps between standard evaluation metrics and actual performance on a robot.

TL;DR

This independent audit shows that standard "one-step-ahead" accuracy is a poor predictor of a simulator's utility; a model can be highly accurate for a single step but fail catastrophically during the long rollouts needed for training. The study finds that while training on multi-step rollouts is essential for stability, even "more accurate" simulators often fail to produce walking policies, highlighting a significant "objective mismatch" in current robot learning pipelines.

What Problem Does This Paper Address?

Training legged robots in traditional physics engines (like MuJoCo or Bullet) is standard, but these engines can be slow or fail to capture complex real-world dynamics. Learned simulators promise to bridge this gap by learning dynamics directly from data.

However, most researchers evaluate these simulators using one-step-ahead prediction error. The problem? A robot policy doesn't use the simulator for one step; it uses it for hundreds of steps (a "rollout"). Small errors at step one can compound exponentially, leading to "hallucinated" states that never occur in reality. This paper asks: Do our current metrics actually predict if a simulator is good enough to teach a robot how to walk?

Methods

The author performed an end-to-end audit by first rebuilding the RWM dynamics model from scratch, ensuring a bit-for-bit match with the original released code’s outputs and gradients.

Figure 1. Checks are run on the simulator; what matters is measured on the policy. The stages this paper follows. Sections 4 to 8 measure the simulator on recorded data. Section 9 measures policies trained inside it, in a physics model. Whether the first kind of measurement predicts the second is the question of Sections 7 and 9.

Key methodological choices included:

  • Verification: A multi-level check (Shapes, Wiring, Indexing, etc.) to ensure the rebuild was perfect before any training began.
  • Comparison of Training Horizons: Testing "teacher forcing" (training only on the next step) against "rollout training" (training on the model's own multi-step predictions).
  • The "Walk Test": Instead of just measuring error, the author built a custom MuJoCo model of the ANYmal D and tested whether policies trained inside various versions of the learned simulator could actually walk in this "ground truth" physics environment.
  • Compression Audit: Testing how 8-bit quantization and weight pruning affect long-horizon stability.

Datasets

  • Released RWM Recording: 10 episodes (200 seconds total) of ANYmal D trotting at 50 Hz. This is a tiny dataset compared to the 6 million transitions typically used, making it a high-stakes test of data efficiency.
  • MuJoCo Synthetic Data: The author generated 6 million transitions from their own hand-built physics model to test if more data could solve the observed failures.

Key Insights and Ideas

  • The One-Step Reversal: A model that looks superior when predicting just one step ahead can be 5.9 times worse than its competitor at a 7-second horizon. One-step checks actively misguide design choices.
  • Stability is Learned, Not Given: Models trained on rollouts learn to "damp" their own errors. In contrast, one-step-trained models tend to amplify errors, leading to a positive Lyapunov growth rate (instability).
  • Uncertainty "Collapse": In the RWM-U variant, individual network variance collapses to near-zero because the loss function used is smallest at zero variance. While ensemble disagreement (the "spread" between multiple models) is a good ranking tool for error, its absolute scale is wildly off, making it a poor "distance to truth" metric.
  • The Accuracy Paradox: In the most surprising finding, fine-tuning a simulator to be more accurate on physics data often broke its ability to produce walking policies. A simulator that only sees "perfect" walking doesn't know what failure looks like, allowing the RL policy to exploit regions of the state space where the simulator is "optimistically wrong."

Key Results

  • Rollout Training Wins: Training on 8-step rollouts beat one-step training on 10/10 held-out episodes at long horizons (368 steps).
  • Compression Deception: Pruning 30% of weights seemed safe one step ahead (retaining 90% skill) but was a disaster at 2 seconds (retaining only 39% skill).
  • Calibration Failure: Ensemble disagreement covered only 4.6% of outcomes at one sigma, compared to the 68% expected for a calibrated estimate.
  • Policy Failure: Every simulator trained from scratch—even those with higher accuracy than the original released model—failed to produce a walking policy. Only the original released model and its lightest compressed copies succeeded.

Limitations and Future Work

The author notes several constraints: the study is limited to one robot (ANYmal D), one gait (trotting), and flat ground. Furthermore, all "ground truth" testing was done in a hand-built MuJoCo model, not on physical hardware.

Future work suggested includes:

  1. Testing the "walk test" on simulators from entirely different architectures.
  2. Training models on data that explicitly includes falls and recoveries to prevent "model exploitation."
  3. Developing pre-deployment checks that don't require a "trusted" policy to run inside them.

Summary updated

Abstract

A legged robot can learn to walk inside a learned simulator: a neural network that predicts the robot's next state and is run forward on its own predictions. They are usually trained and checked one step ahead, although a policy uses them for a hundred steps or more. We audit one published system end to end, without a GPU: the dynamics model of the Robotic World Model (RWM) for the ANYmal D quadruped, and the uncertainty estimate of its follow-up, RWM-U. We rebuilt the model to match the released code's outputs, losses and gradients exactly, measured 101 trained simulators, 200 compressed copies and 38 simulators of three other architectures, and trained policies inside simulators that we tested in a hand-built MuJoCo model of the robot. Training on the model's own rollouts beat one-step training at 368 steps on all ten held-out episodes (registered), although one-step training was better one step ahead. One-step checks therefore misjudge design choices: pruning 30% of the weights kept 90% of the model's skill one step ahead and 39% at 2 s. Ensemble disagreement ranked rollout error well (Spearman +0.90 at 100 steps), yet covered 4.6% of outcomes at one sigma, against 68% for a calibrated estimate, as each network's own variance collapses under the implemented loss. Two checks that need no recorded future ranked some pools of simulators well, but neither was shown better than 8-step error in a primary test. At the robot level, accuracy has not predicted walking: all 18 fine-tuned copies we tested predicted better than the released model one and eight steps ahead, yet 13 gave failing policies, and registered tests of whether any check ranks simulators by their policies were not settled. In a secondary test, running the authors' policy inside each simulator ranked them well. We also report nine of our own claims that we withdrew. Scope: one robot, one gait, simulated data, no hardware.
BibTeX
@misc{singh2026auditing,
  author = {Joyjeet Singh},
  title = {{Auditing a Learned Simulator for Legged Robots: The Robotic World Model on ANYmal D}},
  year = {2026},
  eprint = {2611.23118},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23118},
}
Figure