TriWorldBench: A Tri-View Consistency Perspective on Embodied World Models

Xuanyi Liu, Haofeng Wang, Ruiqi Li, Danni Yu, Rui Wan, Ruixu Zhang, Siyu Tao, Xue Yang, Shaofeng Zhang, Zicheng Zhang, Jiaqi Zhang, Siwei MaarXiv 2026 · arXiv

TL;DR

TriWorldBench is a new evaluation benchmark for embodied world models that shifts the focus from single-view visual quality to multi-camera consistency. By evaluating synchronized video streams from a robot's head and dual wrist cameras across 19 distinct metrics, it ensures that a model's predictions are physically and logically compatible across all viewpoints.

What Problem Does This Paper Address?

Embodied world models are designed to predict how an environment will change when a robot executes an action, serving as a powerful tool for simulation, planning, and training. To operate effectively, modern manipulation robots often rely on a multi-camera setup: a global "head" camera providing workspace context, and "wrist" cameras capturing close-up gripper-object interactions.

Figure 1. Complementary views of one manipulation event. The head camera provides workspace context; the wrist cameras reveal local geometry and gripper–object contact. TRIWORLDBENCH evaluates both the quality of each predicted view and agreement across the three views.

The core issue is that existing benchmarks evaluate these video streams independently. A world model could generate three individually realistic videos that completely contradict one another—for instance, the head view might depict a successful object pickup, while a wrist view shows the gripper missing the object entirely. TriWorldBench addresses this gap by treating multi-view predictions as a single, synchronized triplet, ensuring that what the robot "sees" from its head matches what it "experiences" at its wrists.

Methods

TriWorldBench introduces a comprehensive evaluation protocol that evaluates a synchronized triplet of videos (head, left-wrist, and right-wrist) across 19 metrics grouped into 6 dimensions:

Table 1. The 19 metrics reported by TRIWORLDBENCH, grouped into six dimensions. H, L, and R denote the head, left-wrist, and right-wrist cameras; “H + wrist” denotes a comparison between the head view and the wrist view of the acting arm; “joint” denotes a measurement over the triplet as a whole. H/L/R metrics are computed separately in each view before aggregation. Pixel and structural comparisons use the corresponding ground-truth camera, not a different viewpoint.

Key methodological innovations include:

  • Cross-View Consistency via VLMs: The benchmark leverages Vision-Language Models (VLMs) to perform direct head-to-wrist compatibility checks, verifying whether the views agree on the robot's state, action phase, and object conditions.
  • Reference-Derived Action Phases: Rather than treating a video uniformly, TriWorldBench uses the ground-truth trajectory to segment the task into distinct phases (e.g., idle, approach, grasp, hold, manipulate, release). This allows the benchmark to apply phase-conditioned penalties—preventing models from "cheating" by generating static, frozen frames during high-motion phases.
Figure 2. Reference-derived action phases select synchronized keyframes and head-to-wrist pairs (arcs). They also provide context for evaluating the expected motion in each view.
  • Task-Grounded Visual Quality: To prevent models from scoring highly simply by generating "pretty but wrong" videos, visual quality metrics are automatically discounted if the model exhibits poor trajectory accuracy or low cross-view consistency.

Datasets

The benchmark is built on RoboTwin 2.0, a dataset containing bimanual manipulation tasks.

  • Test Set: 500 episodes spanning 50 distinct bimanual tasks (including pick-and-place, stacking, pouring, handover, and tool use).
  • Validation Set: A separate 100-episode split complete with reference videos and state annotations to support local development.
  • Environment Variations: Tasks are recorded under both clean and domain-randomized background conditions to ensure models are tested against significant appearance variations.

Key Insights and Ideas

  • Consistency $\neq$ Similarity: Because of wide camera baselines, rapid wrist movements, and frequent occlusions, different views should look different. TriWorldBench evaluates whether the views are physically compatible with the same underlying event, rather than checking if they look alike.
  • Camera-Specific Evidence: The benchmark assigns metrics to the cameras best suited to observe them. For example, overall task instruction following and end-effector trajectories are evaluated from the head view, while local contact and geometry are scrutinized across all three views.
  • Granular Failure Diagnosis: Performance is summarized into an aggregate TWB-Score, but the benchmark deliberately retains per-view, head-to-wrist, and joint dimensions. This allows researchers to pinpoint exactly where a world model fails (e.g., identifying if a model has excellent visual generation but poor physical coherence).

Key Results

The paper focuses on establishing the evaluation framework, metric definitions, and the mathematical formulation of the TWB-Score. Specific baseline performance scores for existing world models were not explicitly detailed in the provided text.

Limitations and Future Work

The authors note that the action phase boundaries used to ground the temporal evaluation are derived from heuristic rules (based on gripper opening and end-effector displacement) rather than verified task-success labels. Additionally, because the framework relies heavily on VLMs for cross-view consistency checks, the evaluation itself is subject to the inherent visual reasoning limitations and potential biases of the underlying VLM judges.

Summary updated

Abstract

Embodied world models predict the outcomes of robot actions to support learning and planning. For robots equipped with head and wrist cameras, this requires complementary views: the head view captures the overall task, while wrist views reveal local gripper-object interactions. However, evaluating these views independently cannot determine whether they describe the same action and object state. We introduce TRIWORLDBENCH, a benchmark for evaluating embodied world models through synchronized head, left-wrist, and right-wrist videos. It contains 500 episodes across 50 bimanual manipulation tasks and uses 19 metrics to assess tri-view consistency, task alignment, physical and 3D coherence, motion quality, temporal consistency, and visual quality. By combining cross-view checks with measurements tailored to each camera, the benchmark evaluates whether plausible individual videos also form a consistent prediction of the intended task. We summarize overall performance with TWB-Score and retain per-view results to identify where predictions fail. This extends world-model evaluation beyond single-view visual quality. Code, data, and metric definitions are available at https://github.com/TriWorldBench/TriWorldBench.
BibTeX
@misc{liu2026triworldbench,
  author = {Xuanyi Liu and Haofeng Wang and Ruiqi Li and Danni Yu and Rui Wan and Ruixu Zhang and Siyu Tao and Xue Yang and Shaofeng Zhang and Zicheng Zhang and Jiaqi Zhang and Siwei Ma},
  title = {{TriWorldBench: A Tri-View Consistency Perspective on Embodied World Models}},
  year = {2026},
  eprint = {2609.26314},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.26314},
}
Figure