WanderBench: Can World Models Explore Dynamic Worlds from Video?

Yuhang Chen, Jiheng Li, Jiale Wang, Zai Wang, Guangyue LiuarXiv 2026 · arXiv

TL;DR

WanderBench is a new benchmark designed to evaluate how faithfully video world models can explore dynamic scenes recorded in a source video. The authors find that current models struggle with long-term camera control and "memory" during revisits, prompting the creation of WanderLand, a training-free framework that significantly improves temporal synchronization and scene consistency by anchoring generation to same-instant source evidence.

What Problem Does This Paper Address?

Recent advances in video generation have birthed "world models"—systems that allow users to interactively explore a visual world via camera or action controls. While many models can imagine a world from a single image or text prompt, a more difficult challenge is video-conditioned exploration: taking a real, recorded video and re-observing its events from entirely new perspectives.

The authors identify a critical gap: existing benchmarks primarily evaluate perceptual quality or simple camera following. They don't measure if the model preserves the "truth" of the source video. For instance, does a dancer in the background still perform the same move at the same time when the camera moves? Does a location look the same when the user circles back to it? Current models often "drift," losing sync with the source dynamics or redrawing revisited areas inconsistently.

Methods

The paper contributes two major components: a benchmark and an enhancement framework.

1. WanderBench (The Benchmark)

WanderBench evaluates models across six dimensions using 24 human-validated sub-metrics. Unlike previous benchmarks, it scores exploration against time-synchronized source events.

  • Dimensions: Visual Quality, Camera Control, Source-World Fidelity (identity), Source-Dynamics Fidelity (timing/motion), Spatiotemporal Consistency, and Dynamic Revisit Consistency.

2. WanderLand (The Framework)

WanderLand is a training-free, inference-time enhancement that can be wrapped around existing frozen world models. It uses an evidence-ordered conditioning scheme:

  1. Synchronized Camera: It anchors the commanded camera to the source camera of the same instant rather than just the first frame.
  2. Same-Instant Source: It warps the source frame from the current timestamp into the target view, ensuring moving subjects appear at the correct time.
  3. Static Memory: When a camera revisits a location, it fills "holes" (areas the source video can't see) with the model's own earlier generated content, preventing the scene from being redrawn differently.
Overview of WanderLand's evidence-ordered conditioning

Datasets

The authors introduced the WanderBench dataset, which consists of:

  • 745 single-shot source videos: A mix of real-world footage (dance, film, nature) and AI-generated content (realistic, anime, artistic).
  • Three difficulty tiers: Simple (3–8s), Medium (8–15s), and Hard (15–25s), categorized by the complexity of subject motion and lighting.
  • 5,634 action keys: Paired camera routes (keyboard-style inputs) that include random exploration and "out-and-back" routes to test revisits.
Figure 1: Overview of WanderBench. (a) Data collection and curation. (b) Six evaluation dimen- sions with 24 sub-metrics. (c) Task setting: given a source video and action keys, a model re-renders the scene from novel viewpoints while staying synchronized with the source.

Key Insights and Ideas

  • Visual quality is a deceptive metric: The authors found that a rollout can look "clean" and stable (high visual quality) while completely failing to follow camera commands or source events. Per-frame quality metrics are insufficient for world models.
  • The "Static vs. Dynamic" Trade-off: Models are generally good at preserving the look of a scene (Source-World Fidelity) but struggle significantly with when things happen (Source-Dynamics Fidelity), especially as clips get longer.
  • The Revisit Failure: Most current models have no "memory." When a camera returns to a previously seen spot, the model simply hallucinates a new version of that spot, leading to jarring inconsistencies.
  • Short horizons hide flaws: Frame-limited models often score well on short clips, but the "Hard" tier of WanderBench reveals that these models collapse when tasked with sustained exploration.

Key Results

The authors benchmarked eight state-of-the-art models. The results showed a clear trend: as difficulty increased, camera control and dynamic fidelity plummeted.

Table 2: Results on WanderBench by difficulty tier. Overall is the equal-weight mean of the six dimensions. Bold marks the best score within each tier. Rows of the same model share a colour; grey rows are frame-limited models, which are evaluated only on Simple and only on the frames they can generate.

WanderLand provided consistent improvements across all five models it was applied to:

  • Source-Dynamics Fidelity: Improved by up to 8.3 points (on InSpatio-World 1.5).
  • Dynamic Revisit Consistency: Improved by 2.4 to 4.8 points across all backbones.
  • Zero Trade-offs: Unlike many enhancements, WanderLand improved fidelity without degrading visual quality or camera control.
Table 3: Change brought by WanderLand on each dimension (score with WanderLand minus score without). World-in-World and WorldForge are not evaluated, since both are themselves training-free frameworks built on pretrained world models.

Limitations and Future Work

  • Input Requirements: WanderLand requires access to the model's internal camera and source-render inputs, meaning it cannot be used as a "black box" for models that don't expose these.
  • Geometry Dependence: It relies on the base model's own geometry module; if the model's underlying depth estimation is poor, WanderLand's warping will suffer.
  • Future Directions: The authors suggest that models performing well on WanderBench could eventually enable the generation of full-scene 4D Gaussian Splatting from a single video, moving toward truly immersive, time-synchronized digital twins of real-world events.

Summary updated

Abstract

Video world models conditioned on an input video make it possible to explore real, recorded dynamic scenes, enabling re-shooting of footage, free-viewpoint replay, and real-to-sim environments for embodied agents. Such uses require the generated rollout to stay faithful to the recorded world, yet existing benchmarks start from a single image or a text prompt, and video-conditioned generation is evaluated mainly on perceptual quality and camera following. These evaluations do not directly assess whether exploration preserves subject identity, event progression, and scene consistency upon revisiting a location. We introduce WanderBench, a benchmark for exploring dynamic worlds from video. It comprises 745 single-shot source videos of real and AI-generated content in three difficulty tiers (3-25 s), paired with camera routes given as keyboard actions, and evaluates six dimensions with 24 human-validated sub-metrics. Benchmarking eight models, we find that as clips grow longer, models keep the static scene and visual quality but lose camera control and the source dynamics, and that none keeps revisited places consistent. We further propose WanderLand, a training-free framework that conditions each frame of a frozen model on the same-instant source and, on revisits, on its own earlier static content. WanderLand improves five models on every dimension, by up to 8.3 points in source-dynamics fidelity.
BibTeX
@misc{chen2026wanderbench,
  author = {Yuhang Chen and Jiheng Li and Jiale Wang and Zai Wang and Guangyue Liu},
  title = {{WanderBench: Can World Models Explore Dynamic Worlds from Video?}},
  year = {2026},
  eprint = {2611.12557},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12557},
}
Figure