Comparing Each Generation with Independent Siblings Gives Video World Model Post-Training a Proper Objective for Outcome Frequencies

Xin Li, Yiwen ChenarXiv 2026 · arXiv

TL;DR

Standard post-training for video world models often causes "outcome collapse," where models only generate the most probable future and ignore rare but physically possible ones. This paper introduces Sibling-paired Policy Optimization (SibPO), which uses multiple parallel generations ("siblings") to estimate outcome frequencies and creates a mathematically "proper" objective that forces the model to match the true distribution of physical outcomes.

What Problem Does This Paper Address?

Video world models are increasingly used as simulators for robotics and physical reasoning. However, a single starting frame often leaves the future ambiguous—for example, a falling cloth might hit or miss a target with different probabilities.

The current standard for improving these models is "physics post-training," where generated videos are scored against a single recorded "ground truth" future. While this makes individual videos look more realistic, it creates a major statistical flaw: to maximize its score, the model learns to always produce the most likely outcome. This is known as outcome collapse. If a simulator only ever shows the most probable event, it becomes useless for estimating risks or success rates in complex environments like robotics.

Figure 1. What each generation is compared with decides the outcome frequencies after post-training. The frame leaves open whether the red cloth will touch the yellow object, and 60% of such futures show contact. Each block is the readout of one gen- erated video, and five samples per method are only illustrative. Flow-GRPO compares each generation only with the recorded fu- ture, so the no-contact outcome loses on average and disappears. SibPO also compares each generation with its siblings, the other generations of the same input. Its reward, in the simplest form, subtracts the mismatch with the record (−) and adds the mismatch with the siblings (+), so the rare outcome is kept.

Methods

The authors propose Sibling-paired Policy Optimization (SibPO). Instead of treating each generated video in isolation, SibPO generates a group of "siblings" from the same input frame using independent noise.

The core innovation is a two-term reward function:

  1. Inverse-Frequency Weighting: It rewards agreement with the recorded "ground truth" future, but weights that reward by the inverse of how often the model currently produces that specific outcome (estimated from the siblings). This gives "extra credit" for matching the record when the outcome is rare.
  2. Sibling Correction Term: It subtracts a term based on how much the current generation agrees with its siblings.

Mathematically, this transforms the training objective from a simple linear score into a strictly proper divergence (specifically a Bregman divergence). This ensures that the objective is only maximized when the model's generated distribution exactly matches the true data distribution.

Figure 2. Overview of SibPO on one training group. All G = 8 generations xg share the history h and draw their own noise εg (shared noise is only a comparison setting in Section 6.3). A frozen readout maps each video to a two-cell outcome vector, contact (orange) or no contact (teal). The matrix pairs the eight generations, grouped by outcome, with each other and with the record (left column): a cell takes the outcome color when the two agree (κ = 1) and stays white when they differ. The reward rg of Eq. (4) weights agreement with the record by ωg, which grows with the white cells of row g, and subtracts the sibling term ρg. The baseline bg recomputes the rewards without xg.

Datasets

The researchers evaluated SibPO across a wide range of physical reasoning benchmarks:

  • PisaBench: A benchmark focused on the distribution of landing times for falling objects.
  • Physion: A dataset of contact prediction scenarios (e.g., will the ball hit the block?).
  • Kubric: A simulated environment for throws, bounces, and collisions.
  • PAWBench & CaliBench: Transfer benchmarks used to test how well the model calibrates to physical probabilities it hasn't seen during training.
  • Real-world Robotics: Evaluated using the RT-1 / SIMPLER protocol to see if better world models lead to better robot policy evaluation.

Key Insights and Ideas

  • The "Mode" Trap: The paper proves that any reward function that only compares a generation to a single recorded future is inherently biased toward the mode (the most frequent outcome). This is a fundamental limitation of standard RLHF-style approaches when applied to ambiguous physical futures.
  • Siblings as Frequency Estimators: By using a finite group of siblings, the model can estimate the "inverse frequency" of an outcome without needing to know the true underlying distribution beforehand.
  • Proper Objectives Matter: Unlike "diversity rewards" which often add arbitrary noise to prevent collapse, SibPO derives its objective from the data itself. It doesn't just ask for any diversity; it asks for the correct diversity.

Key Results

SibPO consistently outperformed existing state-of-the-art methods like Flow-GRPO and DPH-RL in maintaining physical diversity:

  • Better Distribution Matching: On PisaBench, SibPO reduced the Kolmogorov-Smirnov (KS) distance (a measure of how much two distributions differ) by 9.1–10.2% compared to the best non-sibling methods.
  • Preserving Rare Outcomes: Qualitatively, while other models would "land" a falling object at a single consistent time, SibPO correctly maintained the spread of landing times seen in the simulator.
  • Robotics Impact: When used to evaluate robot policies, SibPO lowered the error in estimated success rates by 13%.
  • No Quality Trade-off: Crucially, these gains in distribution accuracy came without degrading the visual quality or physical plausibility of individual videos.
Figure 4 shows qualitative results on PisaBench, where SibPO is the only method that maintains the minority outcome (the object still falling) in its generations, matching the simulator's 70/30 split.

Limitations and Future Work

  • Readout Dependency: SibPO requires a "readout" function (e.g., a segmentation mask or a contact detector) to categorize outcomes. While the authors show this can be done with off-the-shelf tools like SAM 2 or CoTracker, the quality of the calibration depends on the accuracy of these readouts.
  • Computational Cost: Training requires generating multiple siblings per step. While the authors note the cost is equivalent to the popular GRPO method, it is still more expensive than simple supervised fine-tuning.
  • Continuous Outcomes: While the theory covers continuous outcomes via kernels, most experiments focused on discrete "cells" of outcomes. Future work could explore more complex, high-dimensional continuous physical distributions.

Summary updated

Abstract

Video world models are used as simulators, so when a first frame leaves several physical outcomes open, repeated generations should produce each outcome as often as it occurs in the data. Physics post-training scores every generation against the single recorded future of its history, which makes single videos more faithful, but the expected score is linear in the model distribution and is maximized by always producing the most likely outcome. The log-likelihood of the recorded outcome is maximized by the data distribution, and its gradient weights agreement with the record by the inverse of how often the model produces the generated outcome, a frequency that other generations of the same history with independent noise, its siblings, can estimate. To train with this weight from a finite group, we propose SibPO, which performs Sibling-paired Policy Optimization: siblings estimate a truncated inverse-frequency weight for agreement with the record, and a second term subtracts the same weight times the agreement with the siblings, which makes the expected objective a strictly proper divergence over outcome cells at every group size. On PisaBench, SibPO lowers the KS distance to simulated landing times by 9.1-10.2% relative to the strongest method outside its reward family on three of four video models without degrading single videos, and it lowers the error of estimated robot policy success rates by 13%.
BibTeX
@misc{li2026comparing,
  author = {Xin Li and Yiwen Chen},
  title = {{Comparing Each Generation with Independent Siblings Gives Video World Model Post-Training a Proper Objective for Outcome Frequencies}},
  year = {2026},
  eprint = {2611.24512},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24512},
}

Loading related content...

Figure