Twin-Corrected Certificates: Bounding Real Robot Success with Few Hardware Trials

Before a learned robot policy is trusted on hardware, someone has to answer a plain question: how often will it succeed on the real robot? Today there are two answers. The first is to trust the simulator, which is cheap but can be badly wrong: in ManiSkill2 a pick-up-the-cube policy succeeds 91% of the time in simulation and 60% zero-shot on the real arm (ManiSkill2). The second is to run dozens of real trials and compute a statistical lower bound. That is honest, but it gives up the main benefit of testing in a virtual prototype, which is avoiding hardware time and hardware risk. We propose a middle path. For every real trial, we replay a "simulated twin" from the same starting configuration under a simulator whose physics parameters were fitted to a few seconds of real data. We then compute the policy's success rate from thousands of cheap simulated rollouts and use the real-minus-twin differences to correct that estimate. The result is a lower confidence bound on real success that stays valid however wrong the simulator is. It gets tighter as the simulator's per-episode predictions agree with reality, so a good simulator buys a sharp certificate from a small number of hardware trials.

Put in two sentences: simulation supplies the sample size, and a few paired real trials supply the honesty. The paired differences also yield a readable measure of the reality gap with two parts, a shift in the average and a loss of per-episode agreement. This tells an engineer whether the simulator is merely biased or truly uninformative for a given policy.

Core Claim

Let $\hat p_{\text{sim}}$ be the mean success over $N$ simulated rollouts. These rollouts draw physics parameters from a distribution fitted to real transitions in the style of offline domain randomization, where "domain randomization" means training and testing across a spread of simulator settings rather than one fixed setting. Let $Y_i \in \{0,1\}$ be the outcome of real trial $i$, for $i = 1,\dots,n$. Let $\hat Y_i \in [0,1]$ be the average success of $K$ simulated twins that start from the reconstructed initial state of that real trial, each using fresh parameter draws. The corrected estimate is

$$\hat p = \hat p_{\text{sim}} + \frac{1}{n}\sum_{i=1}^{n}\left(Y_i - \hat Y_i\right),$$

which is unbiased for the real success rate whenever real initial states are drawn from the distribution the simulator samples. No assumption is made about the simulator's physics. The certificate is a finite-sample lower bound on this quantity; we use an empirical-Bernstein bound on the bounded differences $Y_i - \hat Y_i \in [-1,1]$, plus a small term for the simulated average. Its width scales with the standard deviation of $Y_i - \hat Y_i$, not with that of $Y_i$. This is the same logic as a control variate: when twins predict real outcomes well, the differences have low variance and fewer real trials achieve the same bound. When twins are uninformative, the method degrades gracefully toward the real-only bound instead of failing silently.

What is new is the combination of three things. The bound is absolute and valid, not a ranking. Real trials are paired with state-matched simulated twins. And the parameter distribution that generates the twins is fitted to real data, so the randomized simulator serves as a measurement instrument rather than only a training device. We also make a falsifiable secondary claim: fitted distributions such as DROPO's produce more correlated twins than hand-set uniform ranges, and therefore tighter certificates.

Relation to Prior Work

Two papers motivate the problem from opposite sides. Validate on Sim, Detect on Real ranks domain-randomized policies without hardware rollouts. It combines held-out simulation success with an out-of-distribution score computed on real images, and it improved ranking accuracy in 87% of cases over simulation validation alone. It yields only relative orderings and covers visual randomization only. How Generalizable is My Behavior Cloning Policy? supplies the rigorous side: exact lower confidence bounds on binary success from real trials, released as stochastic_verification. It needs about 50 trials per policy and uses no simulator at all. We keep that paper's validity standard and use simulation to reduce the real-trial count.

Three lines of work supply the simulator that makes twins informative, by fitting physics distributions to small amounts of real data. DROPO fits a mean and variance for each parameter from 13 seconds of real transitions and uses the variance to absorb unmodeled effects. BayesSim infers multi-modal posteriors over parameters. EASI matches transitions adversarially from a single real trajectory. All three stop at better training. None asks how much the fitted simulator can be trusted to predict real outcomes, and none checks that claim statistically. We reuse DROPO as the default fitting method.

Two theoretical results constrain the design. Understanding Domain Randomization bounds the transfer gap only when reality lies inside the randomized family. Provable Sim-to-Real Transfer via Offline Domain Randomization proves consistency of fitted distributions under smooth dynamics, and contact breaks that smoothness. Because our certificate draws its validity from the paired real trials rather than from any modeling assumption, it holds exactly where those guarantees fail, in contact-rich manipulation with structural mismatch. Provably Safe Sim-to-Real Transfer also mixes real and simulated information, but only in a tabular gridworld and with a separability assumption. It certifies safety during learning, whereas we certify success of a fixed policy.

The virtual-prototyping pipelines RoboTwin 2.0 and ROBO-INSTRUCT use simulation to reject bad trajectories or programs. Both acknowledge that passing simulation does not imply real success. RoboTwin's visual failure detector reaches only 0.431 accuracy. Our work quantifies how far "passes in simulation" can be trusted, which is the missing half of pre-verification.

Research Program

The first component is twin reconstruction. For tabletop manipulation, the initial state consists of object poses, robot configuration, and scene layout. We estimate it from a pre-trial camera frame, using pose estimation or fiducial markers during the study phase, and instantiate it in the simulator. We then roll out the same policy $K$ times under fresh draws from the fitted parameter distribution. Reconstruction error is part of what the differences absorb, so it hurts tightness but never validity. We measure how much tightness is lost by comparing twins built from ground-truth poses in sim-to-sim experiments with twins built from estimated poses.

The second component is the certificate itself. We implement the corrected estimator with an empirical-Bernstein bound and a sequential version that lets an engineer stop running real trials once the bound clears a target. For the sequential version we use a confidence-sequence construction, so that stopping early does not invalidate coverage.

The third component is gap diagnosis. The mean of the differences measures bias, meaning the simulator is systematically optimistic. The correlation between $Y_i$ and $\hat Y_i$ measures predictive agreement. Grouping the differences by initial-state features shows where the simulator is wrong. For example, residuals concentrated at high-friction configurations point to a friction range that needs refitting. Feeding these residuals back into the randomization ranges is a natural follow-up, but we treat it as an extension rather than the main claim.

The study proceeds in two stages. The sim-to-sim stage uses a "real" environment that is a structurally modified simulator, so that ground-truth success is known exactly from very large rollout counts. The modifications include an unmodeled asymmetric center of mass as in DROPO, added actuator latency, and a contact model the fitted simulator does not contain. Policies are diffusion and reinforcement-learning policies on robosuite Lift, Can, and Square, using the Vincent et al. evaluation code, plus ManiSkill2 PickCube. Here we can measure true coverage over thousands of repeated certification runs. The hardware stage uses a Franka Panda on Lift (the VSDR setting), on box pushing with an asymmetric center of mass (the DROPO setting), and on PickCube. For each policy, a large reference set of real trials, on the order of 100, estimates the true rate, and certificates are computed from random subsets.

Baselines

The naive reference point is real trials only. We apply the exact lower bound of How Generalizable is My Behavior Cloning Policy? to the same $n$ real trials the twin method sees, with no simulator. This is the strongest valid competitor, because anything valid must beat it on tightness at equal $n$ or match it with fewer trials. For reference, that paper reports a 95% lower bound of 0.641 from 38 successes in 50 real pouring trials. We count our method as beating this baseline if, across the evaluated policies, it reaches the bound the baseline gets at $n=50$ using at most 20 real trials, while keeping empirical coverage at or above 95%.

The strongest existing estimator that uses simulation is simulation-only evaluation under a DROPO-fitted distribution. It has zero real-trial cost but no validity guarantee. We run it with the same simulator, the same fitted parameters, and the same $N$. We report its coverage, which we expect to fail on structurally mismatched tasks, as the 91%-to-60% PickCube gap suggests. We also report its point error. For ranking questions, we additionally compare against VSDR on Lift, using its published protocol. We check whether ordering policies by our lower bound matches or beats its reported 87% improvement over simulation validation. No reported result was found on Lacuna for any absolute real-success bound derived from simulation.

Four ablations isolate the key ingredients. The first removes pairing: twins are replaced by the unconditional simulated mean. This reduces to a real-only estimate plus a constant and should recover the naive width, which confirms that state matching is where the savings come from. The second replaces the fitted distribution with hand-set uniform randomization ranges, which tests whether fitting makes twins more predictive. The third varies the number of twins per trial, $K \in \{1, 8, 64\}$. The fourth uses ground-truth versus estimated initial states. Every method gets identical real-trial subsets, identical simulated compute, and the same fitted simulator. Any settings, such as $K$, are fixed in the sim-to-sim stage and frozen before hardware runs.

Why not just…?

Why not just run 50 real trials and use the exact bound? This is the honest default, and on tasks where the simulator predicts nothing it will be about as good as our method. The claim to test is whether pairing reduces trial counts when the simulator is informative. The hardware results settle this directly: if the certified bound at $n=20$ is not clearly tighter than the real-only bound at $n=20$, the method is not worth its overhead for that task class, and we will report which classes those are.

Why not just trust the fitted simulator? DROPO's fitted distribution cut pushing error from about 10 cm to about 2 cm (DROPO). That makes it good for training, but nothing guarantees its success-rate predictions. The existing theory requires reality to lie inside the randomized family or requires smooth dynamics (Chen et al., Fickinger et al.). Our structurally mismatched sim-to-sim environments measure this failure directly as coverage below the nominal 95%.

Why not just treat simulation results as a Bayesian prior over the real success rate? This is the most serious simple alternative. A prior built from simulation shrinks the estimate toward the simulated rate and yields tight intervals. However, when the simulator is biased, the interval is confidently wrong until enough real data overwhelm the prior, which takes exactly the number of trials we are trying to avoid. We include this as a baseline, with the prior strength tuned on the sim-to-sim stage, and compare coverage under mismatch. If it keeps coverage while being tighter, we will adopt it, but we expect a coverage loss wherever the simulator is systematically optimistic.

Why not just rank policies with VSDR? Ranking answers "which policy is best" but not "is the best one good enough to deploy," and deployment decisions need the second answer. VSDR also covers visual shift only, while our tasks involve physical mismatch. The Lift comparison shows whether an absolute bound also ranks policies competitively.

Decisive Experiments

The first test is coverage. In the sim-to-sim stage, across 1,000 repeated certification runs per policy and mismatch type, the twin-corrected bound must fall below true success at least 95% of the time. The simulation-only and Bayesian-prior baselines are expected to fall short on the latent-mismatch variants. The second test is efficiency. We plot bound width against $n$ for the twin method and the real-only bound, and read off the real-trial savings at a fixed width, with the savings shown as a function of the measured correlation between twin and real outcomes. The control-variate argument predicts that the width shrinks roughly with $\sqrt{1-\rho^2}$, where $\rho$ is that correlation. The third test is hardware replication. On Lift, box pushing, and PickCube with the Franka, bounds from 10, 20, and 30 trials are compared against the 100-trial reference. Fitted and uniform twin distributions are compared to test whether calibrating the simulator improves twin correlation. The fourth test is gap diagnosis. On PickCube, where the published sim-to-real drop is large, we check whether the residuals separate a bias component from a decorrelation component, and whether they point to identifiable initial-state regions. The last check is falsification. If twin correlation on real contact-rich tasks is close to zero even with fitted parameters, the proposal's premise is wrong for that regime, and we will report it as a negative result about how predictive current simulators are for individual episodes.