Abstract
World models have revolutionized video generation and reinforcement learning by encoding complex environments into compact latent spaces. However, their application to rigorous physical systems, such as fluid dynamics or wave propagation, has lacked a solid theoretical foundation. This paper bridges the gap between generative AI and classical control theory, demonstrating that a world model's success depends fundamentally on observability—the ability to reconstruct a system's full state from a history of compressed representations. We prove that under these conditions, physical dynamics are PAC-learnable, and we demonstrate that these models offer superior long-term stability and up to 500x speedups compared to traditional numerical solvers.
Introduction
In recent years, the AI community has seen a surge in "World Models" that predict future states autoregressively. While effective for visual tasks, researchers often struggle to apply them to physical systems due to two primary uncertainties: the impact of state compression ("tokenization") and the required length of historical context for accurate prediction. This research addresses these gaps by framing world models through the lens of System Theory, specifically focusing on whether a system is observable from its latent representations.
Methodology: The World Model Framework
The authors propose a structured framework consisting of three core components designed to learn physical laws from data:
- Tokenization: High-dimensional physical states (e.g., a 2D grid of fluid velocities) are projected into a low-dimensional latent space using patch averaging.
- Latent Dynamics: An autoregressive model predicts the next latent state based on a history of $k$ previous states.
- Super-resolution (Decoder): A module reconstructs the full-resolution physical state from the predicted latent sequence.
The mathematical core of this work (Theorems 4 & 5) proves that if a system is observable, a world model is "Probably Approximately Correct" (PAC-learnable). This provides a guarantee that the model will learn the underlying dynamics given sufficient data and history.
Experimental Setup and Datasets
The framework was evaluated across a hierarchy of dynamical systems to test the limits of the observability theory:
- Linear PDEs: Modified Heat and Wave equations with spatially varying coefficients to ensure observability under patch-averaging.
- Chaotic Systems: The Kuramoto-Sivashinsky Equation (KSE), a benchmark for spatio-temporal chaos.
- Computational Fluid Dynamics (CFD): A 2D Kármán vortex street (flow around a cylinder) at a Reynolds number of 3900, featuring complex turbulent regimes.
Key Insights
The Observability Threshold
A critical finding is the existence of a "history threshold." There is a specific point where adding more historical frames ($k$) leads to a sudden drop in prediction error. This occurs exactly when the history length provides enough information to satisfy the system's mathematical observability requirements. Without this, even massive models fail to learn the dynamics.
Stability and Performance
Unlike Neural Operators (such as FNO or DeepONet), which often diverge during long-term rollouts, the autoregressive world model approach remains stable over hundreds or thousands of frames. By treating physics as a constrained video generation task, the authors show that GAN-based architectures (like NVIDIA’s LongVideoGAN) can capture fine-grained turbulent structures that standard regression models miss.
Results
- Long-Term Accuracy: On the Wave equation at 1000 frames, traditional neural operators like FNO diverged significantly, while the world model maintained a stable MSE.
- Massive Speedup: For the Kármán vortex street, the world model generated frames in 0.032 seconds, compared to 17 seconds for traditional CFD—a 500x speedup.
- Chaos Capture: The model achieved an $L_2$ loss of $4 \times 10^{-6}$ on the chaotic KSE dataset, successfully modeling the onset of chaos after an initial burn-in period.
Limitations and Future Work
While powerful, the theory relies on the existence of a continuous, invertible reconstruction operator. Pathological coefficient choices can violate this, making systems unlearnable. Furthermore, in chaotic regimes, models eventually "desync" from ground truth due to the butterfly effect, where small latent errors amplify over time. Future research will focus on characterizing datasets that satisfy these formal assumptions and handling "near-unobservable" systems where gradient descent struggles despite theoretical feasibility.
Summary updated