Abstract
Modern Image Generation Models (IGMs) often struggle to maintain physical consistency when predicting future states of a scene. This paper introduces "Chain of Time" (CoT), a zero-shot prompting method that encourages models to simulate physics by generating intermediate frames. While CoT significantly improves accuracy in 2D motion and gravity tasks, it also highlights model limitations in inferring latent physical parameters, such as fluid flow rates, which can lead to compounded errors over time.
Introduction
While Vision-Language Models (VLMs) and IGMs like GPT-4o produce visually impressive results, their internal "world models" frequently fail to adhere to basic physical laws. For instance, predicting the trajectory of a bouncing ball often results in physically impossible outputs. This research aims to improve physical simulation performance without expensive fine-tuning and to provide a method for auditing the internal physical reasoning of these models by forcing them to "show their work."
Methodology
The core contribution is Chain of Time (CoT), a prompting strategy inspired by Chain-of-Thought reasoning and the cognitive theory of mental simulation. Instead of a "Direct Prediction" approach—where a model is given initial frames and asked to jump directly to a distant future state—CoT requires the model to generate a sequence of intermediate frames in-context.
As shown in the figure above, the model generates one frame at a time, which is then fed back into the conversation history. This iterative process allows the model to "visualize" the physical progression. To analyze these simulations quantitatively, the researchers used classical computer vision techniques (such as Hough transforms) to extract object coordinates and fluid levels from the generated images, comparing them against ground-truth physical data.
Experimental Setup
The method was evaluated across four distinct physical domains designed to test different aspects of reasoning:
- 2-D Motion: Constant velocity of a rolling ball on a flat surface.
- 2-D Gravity: Parabolic trajectories and acceleration of projectile motion.
- 3-D Fluids: Dynamic filling of a container with water from a pipe.
- 3-D Bouncing: Real-world video sequences of various balls hitting the ground, testing collisions and elasticity.
Results and Discussion
The "Image Chain-of-Thought" proved highly effective for simple 2D physics. In motion and gravity tasks, CoT significantly outperformed direct prediction; for 2D motion, the Root Mean Square Error (RMSE) dropped from approximately 200 to below 100. However, the study revealed a critical weakness: models are sensitive to salient visual cues (like current water levels) but largely "blind" to latent parameters (like flow rate).
In the Fluids domain, CoT actually performed worse than the baseline. Because the model often misestimated the initial flow rate, generating more intermediate steps simply allowed this error to accumulate, leading to a monotonic increase in error as step size decreased. Similarly, in Bouncing tasks, while the model captured the general trajectory, it consistently underestimated physical elasticity, resulting in "lazy" rebounds compared to reality.
Conclusion
Chain of Time demonstrates that forcing image models to visualize intermediate states can significantly enhance physical reasoning for salient tasks. However, the lack of self-correction and the inability to infer hidden physical constants remain significant hurdles. Future work should focus on enabling models to estimate parameters like mass or friction before simulation and exploring CoT's application to higher-level causal reasoning.
Summary updated