PhysLoop: Harnessing LLMs' Physics Awareness for Self-Improving Video Generation Models

Yuxuan Chen, Xiaojie Li, Hao Chen, Juncheng Ma, Kewei Zhang, Zhongjing Du, Guoqiang Wei, Fangfang Wang, Zhen Dong, Zhiyang Dou, Liming Zhao, Yan Zeng, Jianwei Zhang, Daquan Zhou

TL;DR

PhysLoop is a unified "harness" that uses Large Language Models (LLMs) and physics simulators to generate and repair videos with high physical fidelity. By using simulators to provide ground-truth motion and generative models to provide visual realism, it creates a path for video models to autonomously improve their understanding of the physical world.

What Problem Does This Paper Address?

While modern video generation models can produce visually stunning scenes, they are often "physics-blind." They frequently struggle with causal consistency: objects might spontaneously disappear, gravity might behave erratically, or complex interactions—like a chain of dominoes—might fail to follow through.

The core challenge is a data bottleneck. Training models to understand physics requires fine-grained supervision, but collecting high-quality, annotated real-world video is difficult to scale. Conversely, while LLMs possess significant "book knowledge" of physics, they cannot reliably translate that knowledge directly into consistent video pixels. PhysLoop bridges this gap by using LLMs as planners that operate within the "sandbox" of a physics simulator.

Methods

The authors introduce the PhysLoop harness, a multi-agent framework that decomposes video generation into physical planning and visual refinement.

  1. Planner & Asset Manager: An LLM agent (like GPT-6) translates a user request into a detailed task specification and retrieves or generates the necessary 3D assets.
  2. Executive Agent (Simulation): Instead of generating pixels immediately, the system runs a physics simulation using engines like Unreal Engine (UE) Chaos, Taichi, or Genesis. This ensures the motion is governed by actual physical laws.
  3. Verifier Agent: The system automatically checks the simulation data against the task requirements (e.g., "Did the ball actually enter the basket?"). If it fails, the agent receives feedback and revises the scene configuration.
  4. Refiner Agent: Once a physical trajectory is verified, a video-to-video generation model (the "Refiner") takes the low-fidelity simulator render and transforms it into a realistic video, using the simulation as a motion reference.
  5. Final Verification: A final check ensures the visual refinement didn't accidentally break the physics (e.g., by adding an extra object or changing a trajectory).
Overview of the PhysLoop harness and its workflow

This workflow supports both Training-time data construction (creating massive datasets from scratch) and Inference-time video repair (identifying a "broken" segment in an existing video and replacing it with a physically corrected one).

Datasets

The researchers released a significant suite of resources:

  • PhysLoop-100K: A large-scale dataset of 100,000 videos covering diverse physical phenomena (rigid-body dynamics, fluids, cloth, magnetism, and fracture) in realistic scenes.
  • PhysLoop-Gen: A benchmark of 100 complex generation tasks to evaluate how well models follow physical instructions.
  • PhysLoop-Repair: A dataset of 50 tasks specifically for evaluating the localized repair of physical errors in generated videos.
Examples of diverse physical cases in the PhysLoop dataset

Key Insights and Ideas

  • Simulation as an Executable Sandbox: The "aha moment" here is treating the simulator as a validation layer for the LLM. The LLM doesn't need to know how to render pixels; it just needs to know how to set up a physical experiment that the simulator can then "solve."
  • Decoupling Motion and Appearance: By letting the simulator handle the "how it moves" and the generative model handle the "how it looks," the system bypasses the physical hallucinations common in end-to-end video models.
  • Recursive Self-Improvement (RSI): PhysLoop marks a step toward models that can generate their own high-quality training data. By identifying its own failures and using the harness to create "correction pairs," a model can theoretically learn physics without human-labeled data.

Key Results

PhysLoop significantly outperforms standalone state-of-the-art video models in physical accuracy:

  • Task Success: On the PhysLoop-Gen benchmark, PhysLoop achieved an 85% success rate, compared to 73% for the strongest standalone baseline (Seedance 2.5).
  • Video Repair: The results were even more dramatic in repair tasks, where PhysLoop achieved an 86% success rate, while standalone models all scored below 20%.
  • Human Evaluation: Human raters consistently preferred PhysLoop outputs for physical plausibility (scoring 4.06/5) and causal consistency (4.12/5).
  • Qualitative Superiority: As seen in Figure 3, while baselines often fail to complete causal chains (like a domino failing to hit a vase), PhysLoop ensures the entire sequence of events is physically realized.
Qualitative comparison showing PhysLoop successfully completing complex causal chains where baselines fail

Limitations and Future Work

Despite its strengths, PhysLoop is limited by the coverage of its simulators. If a physical phenomenon (like complex chemical reactions or certain soft-body deformations) isn't supported by the underlying engine, the harness cannot generate it.

Additionally, the Refiner Agent can sometimes introduce visual artifacts or slightly drift from the simulated motion during the "beautification" process. The authors suggest that future work should focus on tighter integration between the simulator and the generative model to ensure pixel-perfect alignment with physical laws. Finally, fully automating the "discovery" of model weaknesses to trigger the self-improvement loop remains an open research goal.

Summary updated

Abstract

Video generation models produce visually realistic videos but still struggle with physical dynamics and causal consistency. Learning from generated videos requires fine-grained physical supervision, yet reliable quality judgments, correction pairs, and controlled variations are difficult to obtain. Large language models (LLMs) possess physical knowledge, but struggle to reliably guide video generation models in producing their training videos. We present PhysLoop, a unified harness designed to support physically grounded self-improvement in video generation models. The harness guides and constrains LLMs throughout video generation, selection, and correction, distilling their physical knowledge into physically consistent videos. The framework supports controllable training-data construction and inference-time video repair. By identifying and correcting failures, PhysLoop marks a key milestone toward recursive self-improvement (RSI) in video generation models with limited human intervention. We release 100K videos spanning diverse, realistic indoor and outdoor scenes, along with generation tools and two evaluation datasets: PhysLoop-Gen and PhysLoop-Repair. We evaluate controllable generation and localized repair on these datasets, respectively, and assess physical behavior on the external Physics-IQ Verified benchmark.
BibTeX
@misc{chen2026physloop,
  author = {Yuxuan Chen and Xiaojie Li and Hao Chen and Juncheng Ma and Kewei Zhang and Zhongjing Du and Guoqiang Wei and Fangfang Wang and Zhen Dong and Zhiyang Dou and Liming Zhao and Yan Zeng and Jianwei Zhang and Daquan Zhou},
  title = {{PhysLoop: Harnessing LLMs' Physics Awareness for Self-Improving Video Generation Models}},
  year = {2026},
  eprint = {2611.16748},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16748},
}
Figure