TL;DR
PhysRepair is a new agentic framework that improves the physical realism of AI-generated videos by adopting a "generate–diagnose–repair" workflow. Instead of forcing physical constraints during the initial synthesis, it identifies physical glitches after generation and uses causal reasoning combined with a memory of past successful repairs to surgically fix inconsistencies.
What Problem Does This Paper Address?
While modern video generators (like Sora or Gen-3) produce visually stunning results, they frequently hallucinate "impossible" physics: objects might pass through one another, move before being touched, or ignore gravity.
Existing solutions typically try to bake physics into the model before generation—either through massive amounts of simulation data or complex prompts. However, this often limits the model's creative diversity and fails to catch errors that only emerge during the generation process. The authors identify two core challenges in fixing these "post-hoc" errors:
- What to repair? A simple spatial mask isn't enough; you need to know how a physical failure propagates through time and across different objects.
- How to repair? Different physical failures (e.g., a splashing liquid vs. a bouncing ball) require different tools, simulation settings, and strategies that are hard to pre-define.
Methods
PhysRepair introduces an agentic loop that treats the initial video as a draft to be refined. The system consists of three main stages:
- Evidence-Grounded Diagnosis: The system uses a suite of perception tools (tracking, segmentation, optical flow) to identify exactly where and when a physical law was broken.
- Spatiotemporal Causal Constraints: To decide what to fix, the agent builds an object-centric causal graph. It identifies the first invalid state and traces its "causal descendants." For example, if a ball is saved by a goalkeeper but the net still ripples as if a goal was scored, the agent identifies both the ball's trajectory and the net's motion as needing repair.
- Experience-Augmented In-Context Learning (ICL): To decide how to fix it, the agent uses two types of memory. Short-term memory tracks the current repair attempts, while Long-term memory stores successful strategies from previous, similar physical failures. The agent retrieves these "lessons" to select the right simulation parameters or generation conditions.
As shown in the comparison below, this differs from traditional methods by allowing for localized, adaptive corrections rather than global regenerations.
The final repair is guided by a physics simulator (like PyBullet or Blender) to create a "counterfactual" physical state, which is then used to re-condition the video generator for a final, physically plausible result.
Datasets
The authors evaluated PhysRepair on Physics-IQ Verified, a benchmark containing 198 instances across 66 real-world physical experiments. The benchmark is diverse, covering:
- Solid mechanics (collisions, friction)
- Fluid dynamics (pouring, splashing)
- Optics (reflections, shadows)
- Thermodynamics and Magnetism
Key Insights and Ideas
- Causality over Geometry: You cannot fix physical errors by looking at frames in isolation. By tracing causal effects through a spatiotemporal graph, PhysRepair ensures that "downstream" effects of a collision (like a box moving after being hit) are corrected alongside the collision itself.
- The "Agentic" Advantage: By using an LLM-based agent to orchestrate tools and reason about failures, the system can adapt to a wide variety of physical domains without needing to retrain the underlying video model.
- Memory as a Scaling Law: As the system repairs more videos, its long-term memory grows. The authors found that the number of iterations needed to fix a video decreases over time as the agent learns from its past "experience."
Key Results
PhysRepair consistently outperformed both general-purpose video models and specialized physics-aware methods.
- Quantitative: On the Physics-IQ Verified benchmark, PhysRepair achieved a top score of 43.6, surpassing the strongest baseline (Cosmos3) and significantly improving upon its own backbone model, MiniMax H3 (which scored 39.8).
- Spatiotemporal Accuracy: The most significant gains were in Spatiotemporal IoU (ST), which jumped from 22.8 to 34.7. This indicates that the model is much better at getting the timing of physical events right.
- Qualitative: In ablation studies, the authors showed that without the "Causal Constraints," the model might fix a primary error (like removing a ghost ball) but leave behind secondary errors (like a net that still moves). The full PhysRepair system resolves these dependencies for a "clean" save.
Limitations and Future Work
- Agent Dependency: The system's success is tied to the reasoning capabilities of the underlying multimodal LLM. A "weaker" agent might make incorrect diagnostic or tool-use decisions.
- Inference Cost: Because it is an iterative "generate-diagnose-repair" process, it is computationally more expensive than one-pass generation methods.
- Safety and Provenance: As video repair becomes more realistic, the authors note the increasing difficulty in distinguishing synthetic content from real recordings, highlighting the need for robust digital provenance mechanisms.
Summary updated