Certifying Real-World Robot Policy Success Across Sampling Budgets with Few Hardware Trials

Diffusion and flow policies are robot controllers that produce an action by gradually turning random noise into a motor command. They are now often compressed into one-step versions, because the full multi-step sampler is too slow for closed-loop control. Two things get decided almost entirely in simulation: which sampling budget to deploy, and whether the fast student is as good as its slow teacher. Simulation is a poor judge of this. Simulators assume zero sensor latency and idealized physics, and the latency difference between a 100-step and a 1-step policy is exactly the kind of effect they hide. Hardware trials, meanwhile, are expensive enough that a team rarely runs more than a few dozen per policy. That leaves wide, often unreported, uncertainty about real success rates.

We propose to treat the simulator as a nuisance model rather than as the answer. A nuisance model is an auxiliary prediction that semiparametric statistics lets you plug in and then correct. Concretely, many cheap simulated rollouts, matched to the same scenes and the same random seeds as a small set of hardware trials, give a prediction of each real trial's outcome. A bias-correction term computed from the real trials then removes the simulator's error, which is the sim-to-real gap itself. The result is a confidence interval on the real success rate at every sampling budget. It stays valid however wrong the simulator is, and it gets tighter the better the simulator tracks reality. The hardware trials are allocated adaptively to the scenes and sampling budgets where the simulator is least trustworthy.

Core Claim

Fix a family of policies that share a backbone and differ only in the number of network evaluations used to sample each action. Call the per-action evaluation count NFE (number of function evaluations), with values $k \in \{1, 2, \dots, K\}$. The quantity we want is the vector $\psi_k$, the real-world success rate at budget $k$, together with the differences $\psi_k - \psi_K$ that decide whether acceleration hurts on hardware.

Let $X$ be a scene, meaning the initial object poses and lighting reconstructed for simulation. Let $Z$ be the noise seed sequence fed to the sampler. Let $Y$ be the binary real outcome. Let $R_k$ indicate that a hardware trial was run at budget $k$, chosen with known probability $e_k(X)$. The efficient estimator has the augmented form

$$\hat\psi_k = \frac{1}{n}\sum_{i=1}^{n}\Big[\hat\mu_k(X_i,Z_i) + \frac{R_{k,i}}{e_k(X_i)}\big(Y_i - \hat\mu_k(X_i,Z_i)\big)\Big].$$

Here $\hat\mu_k$ is a cross-fitted predictor of real success built from simulated rollouts at the same $(X,Z)$, and $n$ is the number of candidate scenes. Because $e_k$ is known by design, the estimator is unbiased whatever the quality of $\hat\mu_k$. The semiparametric content lies in its variance:

$$\mathrm{Var}(\hat\psi_k) \approx \frac{1}{n}\Big(\mathrm{Var}[\mu_k(X,Z)] + \mathbb{E}\big[\sigma_k^2(X,Z)/e_k(X)\big]\Big).$$

In this expression $\sigma_k^2$ is the residual variance of the real outcome around the simulator-based prediction. The formula is the standard efficiency bound for a mean under designed missingness. It says that every hardware trial buys precision in proportion to how badly the simulator predicts that trial.

What would be new is the combination of three ingredients. First, outcomes are coupled through shared noise seeds, and one-step policies make that coupling tight. Second, the nuisance model is pooled across sampling budgets, so cheap one-step simulated rollouts help predict the expensive teacher. Third, hardware trials are allocated to minimize the variance term above, weighted by real trial cost.

Relation to Prior Work

The method comes from off-policy evaluation with efficient influence functions. Double Reinforcement Learning builds cross-fitted, doubly robust value estimators from marginalized density ratios. It assumes the logging and target policies share one transition kernel, and it needs tractable action probabilities $\pi(a|s)$. Regularized Targeted Learning adds a targeting step that keeps estimates within valid reward bounds, and it has the same requirements. Both requirements fail here: simulator and hardware have different kernels, and one-step generative policies have no tractable action density. We therefore avoid density ratios over actions entirely. The only importance weight is a design probability we control. We borrow the efficiency theory, not the estimators.

The adaptive allocation of hardware trials follows the logic of Efficient Adaptive Experimentation with Noncompliance, which adapts data collection to shrink the variance of an influence-function estimator. We apply that logic to choosing which scenes and budgets get physical trials.

Two papers motivate the problem. Robot Policy Evaluation for Sim-to-Real Transfer documents a 24–30% performance collapse on transfer and a 30–50% drop under lighting or camera perturbations. It proposes perturbation tiers and "Success Performance Matching" alignment metrics, but no bias-corrected estimator and no confidence intervals. It also does not test one-step policies. Mean Flow Policy with Instantaneous Velocity Constraint (MVP) builds a one-step action generator, reports multi-step latencies around 113–117 ms, and selects actions with a heuristic Best-of-N critic. It is evaluated only in simulation (Robomimic, OGBench), and its authors name hardware validation as open.

On the supply side, one-step generators such as Mean Flows learn an average velocity over an interval, so that one network call maps noise directly to output. This deterministic, nearly straight noise-to-action map is what makes seed-coupling plausible. Real-to-sim pipelines such as Reconciling Reality through Simulation supply the scene reconstruction that matches $X$ across domains. We use them as inputs, not as substitutes for correction.

Research Program

The first component is the coupled nuisance model. For each candidate scene we reconstruct a simulator twin and roll out each budget-$k$ policy under many seed sequences. We then fit $\hat\mu_k(X,Z)$ as a classifier of real success. Its inputs are scene features, the seed, the simulated outcome and trajectory summaries, and the measured hardware latency of budget $k$ injected into the simulator. It is trained with K-fold cross-fitting on the real trials so that the correction term stays honest.

A single model is shared across $k$. Simulated one-step rollouts are roughly $K$ times cheaper than teacher rollouts, so most simulated data comes from the student. The model learns how real outcomes shift with budget.

We test one hypothesis directly. Under the same seed, a one-step student's real and simulated action sequences stay close for longer than a stochastic multi-step sampler's do, because the student has no per-step noise injections. If this holds, the residual $\sigma_k^2$ is smallest exactly where acceleration is deployed.

The second component is the design. Given current estimates of $\sigma_k(X)$ and the per-trial cost $c_k$, the variance bound is minimized by Neyman-style allocation, $e_k(X) \propto \sigma_k(X)/\sqrt{c_k}$. This allocation follows from minimizing $\mathbb{E}[\sigma^2/e]$ under a budget on $\mathbb{E}[c\,e]$. We run the allocation in batches, re-estimating $\sigma$ between batches. Batching at a fixed design in each round, with known probabilities, keeps the estimator's unbiasedness intact.

The third component is inference. We report simultaneous intervals over $\psi_1,\dots,\psi_K$ and the differences from the teacher using the joint influence-function covariance. We check their coverage empirically.

Baselines

The naive reference point is the hardware-only estimate: the sample mean of real successes at each budget with Wilson intervals, using uniformly allocated trials. This is what most robot papers report. Any proposal must give shorter intervals at equal coverage and equal real-trial count to justify its machinery.

The simulation-only reference is to report simulated success, possibly after IsaacLab perturbation tiers, as the estimate of real success. This is the de facto practice of MVP. We measure its bias against ground truth. We expect it to be narrow and miscovering.

The strongest existing competitor for this question is the calibration approach of Robot Policy Evaluation for Sim-to-Real Transfer: align simulated and real success via Success Performance Matching, then read off a corrected real estimate. We implement it with the same simulated rollouts and the same real trials, and we add a bootstrap interval so it can be scored on coverage. No reported result was found on Lacuna for the interval width or coverage of any estimator of real robot success on these benchmarks. The comparison therefore has to be run by us. Beating it means matching its point accuracy while achieving nominal coverage, which we expect calibration to lose under perturbation tiers where the gap is scene-dependent.

A further baseline is the plain augmented estimator with a simulator-success predictor that ignores seeds and pools nothing across budgets, run with uniform allocation. This is the "prediction-powered" special case. Three ablations then add one ingredient each, isolating the contribution of each component: seed coupling, cross-budget pooling, and adaptive allocation. All methods receive identical scenes, identical simulated compute measured in network evaluations, identical real-trial budgets, and the same hyperparameter search budget for the nuisance classifier.

Why not just…?

Why not just run more hardware trials? That is always valid, and it is the right comparison. The question is the exchange rate. The variance formula says the gain over hardware-only is the fraction of outcome variance the simulator explains. If seed-coupled simulation explains little of the real outcome, which can happen under large unmodeled contact dynamics, our method collapses to the naive estimator at no loss. The experiments report this effective-sample-size ratio per perturbation tier, so a reader can see where simulation is worth having.

Why not just use prediction-powered inference with the simulator as the predictor? This is a real threat, because our estimator is that idea in its simplest form. Our claim is only that the predictor and design matter a great deal for robots. Seed coupling exploits the deterministic one-step map. Cross-budget pooling uses cheap student rollouts to inform teacher predictions. Allocation targets scenes where the simulator fails. The ablation ladder settles this. If the plain variant matches the full method, the honest conclusion is that off-the-shelf prediction-powered inference suffices, and we would report that.

Why not just close the gap with better simulation, via real-to-sim-to-real tuning? Methods like Auto-Tuned Sim-to-Real Transfer and Reconciling Reality through Simulation shrink the gap but do not certify the residual. They complement our approach: a better simulator lowers $\sigma^2$ and narrows our intervals. We test this by running our estimator with nominal and tuned simulator twins and reporting the change in interval width.

Why not just use doubly robust off-policy evaluation from logged data? Double Reinforcement Learning requires action densities that one-step generators lack and a shared transition kernel that sim and real do not share. Estimating densities of implicit policies in continuous action spaces would add a nuisance with slow convergence and extreme ratios. We avoid this by controlling the design.

Proposed Experiments

The first study is semi-synthetic, so that ground truth is known. We take Robomimic and OGBench tasks from MVP and train a flow-matching teacher plus MVP and MeanFlow-style students at budgets 1, 2, 4, 8, and 32 or more. The nominal simulator plays "sim." IsaacLab perturbation tiers V1–V5 from Robot Policy Evaluation for Sim-to-Real Transfer, plus injected per-budget latency matching the reported 113–117 ms multi-step figures, play "real." Ten thousand "real" rollouts per budget give ground truth. We then simulate scarce-hardware regimes of 20 to 200 "real" trials and measure coverage, interval width, and bias across 500 repetitions.

The decisive checks come from this study. Nominal coverage must hold at every tier, including V5 where the sim prediction is worst. Interval width must shrink relative to hardware-only by a measurable factor at tiers V1–V3. The estimated latency-quality curve must correctly rank budgets where simulation alone ranks them wrongly. We also test the seed-coupling hypothesis directly by comparing $\sigma_k^2$ between one-step and multi-step samplers.

The second study is a small hardware validation on a tabletop manipulation task. We use roughly 150 real trials spread over budgets, held against a reserved block of 300 uniformly allocated trials as a reference. The question is whether the estimator's interval for the teacher-student gap contains the reference estimate, and how many trials it saved. Where the DECO-50 real teleoperation data from DECO can train a policy for our platform, we use it to test transfer to a real-data-trained policy, without claiming its humanoid setup.

A negative outcome would also be informative. If coupled simulation explains under roughly 10% of real outcome variance on hardware, the practical message is that sim results should not be reported as robot success without hardware intervals. That finding is itself worth publishing.