How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop

Haojia Gao, Yue Cao, Yumin Liu, Haohua Que, Mingkai Liu, Guanqiu Guo, Zhuo Zou, Bokui Chen, Xueqian WangarXiv 2026 · arXiv

TL;DR

This paper reveals that standard methods for evaluating "Best-of-N" selection in World Action Models (WAMs) are heavily biased by future randomness, a phenomenon known as the "optimizer's curse." By introducing a denoised counterfactual protocol, the authors show that real "selectable headroom" comes from structured action perturbations rather than native policy samples, and they leverage this to build a fail-gated selector that significantly improves closed-loop robot task success.

What Problem Does This Paper Address?

World Action Models (WAMs) are exciting because they don't just predict actions; they predict the visual consequences of those actions. This enables test-time selection: a robot can "imagine" several possible futures for different candidate actions and execute the one that looks most successful.

However, how much can this actually help? Previous studies measured "oracle gain" by executing multiple candidates once and picking the best outcome. The authors argue this is a flawed metric. Because the future is stochastic, the candidate that succeeds in a single trial might just be the "luckiest" one, not the best one. This conflation of candidate quality with continuation randomness leads to an overestimation of the selectable headroom—the actual performance gain a real-world selector could achieve.

Methods

The authors propose a denoised counterfactual protocol to isolate true candidate quality from noise. Key components include:

  • Bitwise-Reproducible Branching: Using simulator snapshots (MuJoCo) to ensure that every candidate action starts from the exact same state, down to the last bit of the random number generator.
  • Repeated Continuations: Instead of running a candidate once, they run it multiple times ($J$ continuations) with different future seeds.
  • Held-out Oracles: They use a "double estimator" approach—selecting the best candidate based on one set of continuations and scoring it on a completely fresh, held-out continuation. This eliminates the "optimizer's curse."
  • Fail-Gated Selector: They design a practical selector that uses a failure gate to decide when to deviate from the base policy (only when failure is predicted) and a ranker to decide where to go among structured perturbations (e.g., shifting the action along coordinate axes).
Overview of the denoised measurement protocol and the fail-gated selector design.

Datasets

The study evaluates four different WAMs (Cosmos Policy, Flex-π, Fast-WAM, and Streaming-WAM) across two major benchmarks:

  • RoboCasa: A large-scale simulation for everyday kitchen tasks.
  • LIBERO-Plus: A benchmark focused on lifelong learning and robustness to object layouts and robot initial states. The authors also release HeadroomBench, a package containing their protocol and over 100,000 branched rollouts for future research.

Key Insights and Ideas

  • The "Optimizer's Curse" is Real: In native policy samples, up to 98% of the variation between candidates is just "continuation luck." Standard estimates overstate the selectable headroom by 1.5x to 4x.
  • Native Samples vs. Structured Perturbations: Sampling more "native" actions from the policy provides almost no real headroom because the samples are too similar. However, structured perturbations (e.g., shifting the gripper position by 2-10cm) create significant, realizable headroom, especially in states where the base policy is likely to fail.
  • Value Heads are Blind to Sibling Actions: Standard value heads are great at judging if a state is good, but they are surprisingly bad at ranking "sibling" actions from the same state. They miss the fine-grained signal present in the model's own imagined futures.
  • Imagination is High-Fidelity: Probes show that the information needed to rank actions is present in the WAM's imagined futures with no detectable loss compared to "real" futures. The bottleneck isn't the world model's prediction; it's the readout (the value head).
Comparison of headroom for native candidates vs. structured perturbations. Note how the held-out oracle (blue) collapses for native samples but remains high for structured ones.

Key Results

  • Closed-Loop Success: The fail-gated selector improved task success from 76.9% to 81.6% on training tasks and from 62.6% to 69.8% on entirely unseen tasks.
  • Targeted Gains: The largest improvements occurred exactly where the policy was weakest. In "robot initial state" perturbations, success jumped by 10.7 points.
  • Efficiency: By using a failure gate, the selector only triggers expensive "imagination" passes at about 35-45% of decision points, making it much more computationally efficient than a naive Best-of-N approach.
A qualitative example of the selector "rescuing" a failure on an unseen task.

Limitations and Future Work

  • Simulation vs. Reality: The study relies on bitwise-reproducible branching in simulation. While the selector doesn't need this at deployment, the measurement of headroom on physical robots would be limited by reset noise.
  • Heuristic Proposals: The "structured perturbations" used (axis offsets) are simple heuristics. Future work could involve training "advantage heads" or proposal networks to generate even more effective candidate actions.
  • Compute Trade-offs: While the fail-gate helps, test-time selection still adds latency. Optimizing the WAM's imagination speed remains a critical path for real-time robotics.

Summary updated

Abstract

World action models (WAMs) predict the visual consequences of the actions they generate, which makes test-time selection a natural way to improve a fixed policy: imagine the futures of candidate actions and execute the best one. The benefit of selection is bounded by the selectable headroom, how much better the best candidate truly is, but the standard estimate, which executes each candidate once and keeps the best outcome, conflates candidate quality with the randomness of everything that follows. We introduce a denoised counterfactual protocol, based on bitwise-reproducible branching, repeated continuations, held-out oracles, and common random numbers, that measures the headroom a selector can realize, and use its measurements to build a fail-gated selector. Across four WAMs, two benchmarks, over 100,000 branched rollouts, and pre-registered tests, we find that (i) the standard estimate overstates the selectable headroom of native samples: bounds from 17 continuations per candidate place its selectable share between 24% and 66% for three WAMs; (ii) structured action perturbations create up to 16 points of selectable headroom; and (iii) in both WAMs with action-conditioned imagination, imagined futures carry candidate-outcome information with no detectable loss relative to real futures, information that the value head of Cosmos Policy leaves unused. Deviating from the policy chunk only when a readout of the WAM predicts failure, the selector raises closed-loop task success from 76.9% to 81.6% on held-out episodes of training tasks, with the largest gain where the policy fails most often, and from 62.6% to 69.8% on unseen tasks.
BibTeX
@misc{gao2026how,
  author = {Haojia Gao and Yue Cao and Yumin Liu and Haohua Que and Mingkai Liu and Guanqiu Guo and Zhuo Zou and Bokui Chen and Xueqian Wang},
  title = {{How Much Can Best-of-N Help a World Action Model? Measuring Selectable Headroom and Realizing It in Closed Loop}},
  year = {2026},
  eprint = {2610.37196},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.37196},
}
Figure