When do World Models Successfully Learn Dynamical Systems?

Edmund Ross, Claudia Drygala, Leonhard Schwarz, Samir Kaiser, Francesca di Mare, Tobias Breiten, Hanno Gottschalk

Abstract

World models have revolutionized video generation and reinforcement learning by encoding complex environments into compact latent spaces. However, their application to rigorous physical systems, such as fluid dynamics or wave propagation, has lacked a solid theoretical foundation. This paper bridges the gap between generative AI and classical control theory, demonstrating that a world model's success depends fundamentally on observability—the ability to reconstruct a system's full state from a history of compressed representations. We prove that under these conditions, physical dynamics are PAC-learnable, and we demonstrate that these models offer superior long-term stability and up to 500x speedups compared to traditional numerical solvers.

Introduction

In recent years, the AI community has seen a surge in "World Models" that predict future states autoregressively. While effective for visual tasks, researchers often struggle to apply them to physical systems due to two primary uncertainties: the impact of state compression ("tokenization") and the required length of historical context for accurate prediction. This research addresses these gaps by framing world models through the lens of System Theory, specifically focusing on whether a system is observable from its latent representations.

Methodology: The World Model Framework

The authors propose a structured framework consisting of three core components designed to learn physical laws from data:

  1. Tokenization: High-dimensional physical states (e.g., a 2D grid of fluid velocities) are projected into a low-dimensional latent space using patch averaging.
  2. Latent Dynamics: An autoregressive model predicts the next latent state based on a history of $k$ previous states.
  3. Super-resolution (Decoder): A module reconstructs the full-resolution physical state from the predicted latent sequence.
The proposed world model pipeline, illustrating the mapping from physical state history to latent sequences and back to reconstructed physical states.

The mathematical core of this work (Theorems 4 & 5) proves that if a system is observable, a world model is "Probably Approximately Correct" (PAC-learnable). This provides a guarantee that the model will learn the underlying dynamics given sufficient data and history.

Experimental Setup and Datasets

The framework was evaluated across a hierarchy of dynamical systems to test the limits of the observability theory:

  • Linear PDEs: Modified Heat and Wave equations with spatially varying coefficients to ensure observability under patch-averaging.
  • Chaotic Systems: The Kuramoto-Sivashinsky Equation (KSE), a benchmark for spatio-temporal chaos.
  • Computational Fluid Dynamics (CFD): A 2D Kármán vortex street (flow around a cylinder) at a Reynolds number of 3900, featuring complex turbulent regimes.

Key Insights

The Observability Threshold

A critical finding is the existence of a "history threshold." There is a specific point where adding more historical frames ($k$) leads to a sudden drop in prediction error. This occurs exactly when the history length provides enough information to satisfy the system's mathematical observability requirements. Without this, even massive models fail to learn the dynamics.

Stability and Performance

Unlike Neural Operators (such as FNO or DeepONet), which often diverge during long-term rollouts, the autoregressive world model approach remains stable over hundreds or thousands of frames. By treating physics as a constrained video generation task, the authors show that GAN-based architectures (like NVIDIA’s LongVideoGAN) can capture fine-grained turbulent structures that standard regression models miss.

Mean Squared Error (MSE) evolution for the Heat and Wave equations over 100 frames, demonstrating the model's performance characteristics.

Results

  • Long-Term Accuracy: On the Wave equation at 1000 frames, traditional neural operators like FNO diverged significantly, while the world model maintained a stable MSE.
  • Massive Speedup: For the Kármán vortex street, the world model generated frames in 0.032 seconds, compared to 17 seconds for traditional CFD—a 500x speedup.
  • Chaos Capture: The model achieved an $L_2$ loss of $4 \times 10^{-6}$ on the chaotic KSE dataset, successfully modeling the onset of chaos after an initial burn-in period.

Limitations and Future Work

While powerful, the theory relies on the existence of a continuous, invertible reconstruction operator. Pathological coefficient choices can violate this, making systems unlearnable. Furthermore, in chaotic regimes, models eventually "desync" from ground truth due to the butterfly effect, where small latent errors amplify over time. Future research will focus on characterizing datasets that satisfy these formal assumptions and handling "near-unobservable" systems where gradient descent struggles despite theoretical feasibility.

Summary updated

Abstract

In this work, we explore the use of compact latent representations with learned time dynamics ('World Models') to simulate physical systems. Drawing on concepts from control theory, we propose a theoretical framework that explains why projecting time slices into a low-dimensional space and then concatenating to form a history ('Tokenization') is effective at learning dynamical systems, and characterise when the underlying dynamics admit a reconstruction mapping from the history of previous tokenized frames to the next, with the key novel insight being observability. To validate these claims, we develop a sequence of models with increasing complexity, starting with least-squares regression and progressing through simple linear layers, shallow adversarial learners, and ultimately full-scale generative adversarial networks (GANs). We evaluate these models on a variety of datasets, including modified forms of the heat and wave equations, the chaotic regime 2D Kuramoto-Sivashinsky equation, and a challenging computational fluid dynamics (CFD) dataset of a 2D Kármán vortex street around a fixed cylinder, where our model is successfully able to recreate the flow. Comparisons to FNO and DeepONet show comparable short- and improved long-term accuracy.
BibTeX
@misc{ross2025when,
  author = {Edmund Ross and Claudia Drygala and Leonhard Schwarz and Samir Kaiser and Francesca di Mare and Tobias Breiten and Hanno Gottschalk},
  title = {{When do World Models Successfully Learn Dynamical Systems?}},
  year = {2025},
  eprint = {2507.04898},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.04898},
}
Figure