Robotics and autonomous-driving papers usually report how well a system works in simulation, sometimes add a handful of real trials, and attach error bars computed as if every frame or time step were an independent coin flip. Two errors compound here. Steps within one real episode are strongly correlated, so per-step error bars are too narrow. When studies of human listening tests resample whole groups instead of individual ratings, the measured variability is 1.67 to 2.83 times larger than the naive estimate (MOS vs. AB). Meanwhile, the simulator's numbers say little about reality. On S2R-Bench, real LiDAR detection under glare was 85% worse than simulation predicted. Once the real episode is treated as the honest unit of evidence, a typical study has only 10 to 40 independent data points. In that regime, textbook confidence intervals undercover, and distribution-free bounds such as Hoeffding's are vacuous: about 300,000 samples are needed for a ±0.03 interval on a 5-point scale, versus about 1,000 under the normal approximation (tail-probability analysis of MOS).
This note proposes a confidence bound on real-world performance with three properties. It treats each real episode as one observation. It uses a large pool of simulated rollouts as side information, by pairing each real episode with a simulated replay of the same configuration and bounding only the real-minus-simulated residual. Its leading constants are chosen for the 10–40-episode regime, not for asymptotic elegance. The claim is that this combination yields intervals that keep their stated coverage and are narrow enough to be useful with a few dozen real trials. No single ingredient achieves both.
Core Claim
Let $r_i \in [0,1]$ be the real-world score of episode $i$ (success, or a per-episode averaged detection metric). Let $s_i$ be the score of a simulated replay of the same configuration, for $i = 1,\dots,M$ real episodes. Let $\bar s_N$ be the mean simulated score over $N \gg M$ simulated configurations drawn from the deployment distribution. The target is the real mean $\mu_R$. The estimator is
$$\hat\mu_R = \lambda\,\bar s_N + \frac{1}{M}\sum_{i=1}^{M}\big(r_i - \lambda s_i\big),$$
where $\lambda$ is a weight on the simulator, fixed before the real residuals are examined. The estimator is unbiased for $\mu_R$ regardless of how wrong the simulator is. The simulator only shrinks the variance of the residuals $d_i = r_i - \lambda s_i$, and it can never introduce bias. This is the standard control-variate idea, applied at the episode level.
The non-obvious part is the uncertainty. The residuals $d_i$ are often low-variance, but their range is still about 2. A variance-adaptive bound such as empirical Bernstein has a width of roughly
$$\sqrt{\frac{2\hat\sigma^2 \log(2/\delta)}{M}} + \frac{c\, b \log(2/\delta)}{M},$$
where $\hat\sigma^2$ is the sample variance of $d_i$, $b$ is their range, $\delta$ is the allowed failure probability, and $c$ is a constant equal to $7/3$ in the commonly used form. At $M = 20$ and $\delta = 0.05$, the second term alone is close to 1, so the bound is vacuous on a $[0,1]$ metric even when the simulator is nearly perfect. In the regime that matters, the lower-order constant, not the rate, decides whether the certificate exists. That makes constant-factor refinement the core technical problem here, not a cosmetic one.
The proposal derives and tests episode-level bounds with optimized lower-order terms. Two families are considered: wealth-based "betting" constructions, which build the interval from a sequence of wagers on the residuals, and Bernstein bounds whose constants are tuned for small $M$. For each, the work quantifies the minimum number of real episodes needed for a non-vacuous certificate as a function of how well the simulator tracks reality.
Relation to Prior Work
Several lines of work motivate the episode-as-unit choice by showing that ignoring grouping inflates confidence. The MOS vs. AB study and the biometric significance test show this for listeners and persons. The repeated-measures AUC work shows it for patients. Online bootstrap for policy evaluation shows it for Markov trajectories. All of these correct the variance, but only asymptotically or by bootstrap. None addresses the few-cluster regime or uses a simulator as side information.
The SAA with dependent data paper constrains the design. It shows that discarding correlated samples by thinning or restarting loses information relative to using the whole path. This is why the proposal averages within an episode rather than subsampling it.
Safe-RL certification work supplies the closest competitors. PAC-Bayes control already treats environments as the i.i.d. unit and obtains a non-vacuous hardware bound: 88.6% certified versus about 90% observed on a Parrot drone. This does not contradict the proposal. It confirms that episode-level units give honest bounds. The difference is that PAC-Bayes control certifies generalization of a trained stochastic policy and uses no paired simulator information when certifying deployment. Here the policy is fixed, and the question is how few real trials suffice once simulation is used as a control variate.
Provably Safe Sim-to-Real Transfer certifies where a tabular simulator is accurate. It assumes the simulator is either exactly right or wrong by a known margin, and it treats transitions as independent. The proposed bound assumes neither. POSTPI and Safe Flow Q-Learning compute high-confidence or conformal margins in Safety Gymnasium, but only from simulated rollouts assumed independent or exchangeable.
Constant-sharpening work such as offset Rademacher localization supplies the general attitude: refine the non-asymptotic multipliers rather than the rate. This proposal applies that attitude to mean estimation with very few samples, not to regression.
Research Program
The theory component has three parts. First, it derives finite-sample bounds on $\mu_R$ from the episode residuals, covering empirical Bernstein with optimized constants and a betting construction. For each, it states the explicit width as a function of $M$, $\hat\sigma^2$, and $\delta$. Second, it handles the choice of $\lambda$ without data snooping. Three options are compared: fixing $\lambda=1$; estimating it on a held-out split of real episodes; or updating it predictably episode by episode inside the betting construction, which preserves validity. Third, it gives a "minimum real episodes" formula. This turns the simulator–reality correlation, which can be measured on a pilot set, into the number of real trials needed for a target width. This is the practically useful deliverable.
The empirical component builds paired sim–real evaluation sets in two domains where pairing exists. S2R-Bench provides frame-aligned real and synthetically corrupted versions of 151 driving sequences. Each sequence is a cluster, and the synthetic version serves as $s_i$.
The VoiceMOS 2022 corpus, with 175 systems and about 227 ratings each, provides human ratings as "reality" and an automatic MOS predictor as the cheap "simulator." Utterances and listeners are the clusters. This domain depends on rater identifiers being recoverable from the corpus; if they are not, utterance-level clustering is used alone.
A third, fully controlled testbed uses Safety Gymnasium. Its nominal dynamics serve as the simulator. A perturbed copy, with altered friction, actuator delay, and injected autocorrelated sensor drift, plays the role of reality. This is still simulation, and it is reported as such. Its value is a known ground truth and the ability to dial the simulator–reality correlation up or down.
Baselines
The naive reference point is current practice. It computes a normal-approximation interval over all frames or time steps of the real episodes, treating them as independent and ignoring the simulator. We expect it to undercover badly. The MOS vs. AB result, where variability was underestimated by factors of 1.67 and 2.83, predicts the size of this failure. The bound beats it if it holds nominal coverage where the naive interval does not.
The strongest practical competitor is the cluster bootstrap on real episodes only: resample whole episodes and take percentile intervals. Cluster-based resampling is reported to closely match observed replication variability in the TTS setting (MOS vs. AB). That paper gives no coverage figure, and there is no reported result on Lacuna for cluster-bootstrap coverage on S2R-Bench or with fewer than 40 clusters.
A second strong competitor is the cluster bootstrap applied to the same control-variate residuals $d_i$. This isolates whether finite-sample validity, rather than the simulator correction, is what matters.
For valid but loose references, the comparison uses two bounds on the $r_i$ alone. One is exact Clopper–Pearson for binary success. The other is Hoeffding at the episode level, the method that needs about 300,000 samples for ±0.03 in the tail-probability study.
For robotics certificates, the reference number is PAC-Bayes control's 88.6% certified versus about 90% empirical success on hardware (PAC-Bayes control). This gap of roughly 1.4 points is used as a qualitative target for tightness. It is not a direct comparison, because that bound certifies a different quantity.
Ablations remove one ingredient at a time. They drop the simulator ($\lambda=0$), replace the optimized constants with standard Hoeffding or the textbook empirical-Bernstein constants, and use frames instead of episodes as the unit.
Every method receives identical real episodes, identical simulated pools, the same $\delta=0.05$, and no tuning on test splits. Any $\lambda$ estimation happens on held-out episodes for all methods that use it. The proposal beats a baseline if two things hold. First, its empirical coverage is at least 95% across all $M$ from 5 to 40, where bootstrap variants fall below. Second, its median width is smaller than Clopper–Pearson or Hoeffding by a margin that grows with simulator–reality correlation.
Why not just…?
Why not just use the cluster bootstrap on the real episodes? This is the real threat. If it holds 95% coverage at $M \approx 15$, the finite-sample machinery is unnecessary for validity. The bootstrap with very few clusters is expected to undercover, because the resampled spread underestimates the population spread when $M$ is small. The coverage-versus-$M$ curves below test this directly. If the bootstrap on residuals covers adequately down to $M=10$, the conclusion will be that the gain comes from the simulator correction alone, and the constant-optimization claim will be narrowed accordingly.
Why not just thin the sequence, or restart episodes, to restore independence? Restarting is exactly what the episode-level unit already does. Thinning frames within an episode throws away information without removing between-episode correlation. The SAA dependent-data study found single-path use strictly better than thinning and replication. The proposal keeps every frame by averaging it into its episode score.
Why not just use Clopper–Pearson or Hoeffding on real successes? These are valid. They are also already near-optimal for $r_i$ alone, so constant tuning there gains little. Their weakness is that they cannot exploit a good simulator: the width at $M=20$ stays around ±0.2 no matter how well simulation predicts reality. The ablation with $\lambda=0$ measures exactly how much the simulator buys. If the simulator–reality correlation on S2R-Bench is too weak to shrink width, that ablation will show it, and the honest conclusion will be that the paired simulator does not help there.
Why not just randomize the simulator more? Domain randomization changes the policy, not the certificate. S2R-Bench shows that synthetic corruptions miss structured real effects, so no amount of simulated evidence alone can bound real performance. The proposed estimator stays unbiased even when the simulator is badly wrong.
Proposed Experiments
The decisive check is a resampling coverage study. On S2R-Bench, the population is all 151 sequences, scored per sequence by a fixed fusion detector on real data. The estimators are run on random subsets of $M \in \{5, 10, 20, 40\}$ real sequences, while all synthetic sequences are available as the simulated pool. Each configuration is repeated thousands of times, recording coverage of the population mean and median interval width. The same protocol runs on VoiceMOS at the system level, treating the human mean as truth. It also runs on the Safety Gymnasium perturbed-dynamics testbed, where reality can be simulated to arbitrary depth and the correlation between nominal and perturbed returns is swept from near zero to near one.
The proposal succeeds if two conditions hold. Coverage must stay at or above nominal for all $M$, while frame-level intervals and few-cluster bootstraps fall below. Width must fall below Clopper–Pearson by an amount that tracks the measured correlation, as the minimum-episodes formula predicts.
A second check tests that formula's predictions against the observed $M$ needed for a ±0.1 interval in each domain. It fails if the prediction is off by more than a factor of two.
A final stress test targets the S2R-Bench glare and misalignment subsets, where the simulator is worst. There, the bound should widen gracefully toward the real-only bound and never lose coverage. This is the property that separates the method from approaches that assume the simulator is right.
References
- MOS vs. AB: Evaluating Text-to-Speech Systems Reliably Using Clustered Standard Errors
- S2R-Bench: A Sim-to-Real Evaluation Benchmark for Autonomous Driving
- Analysis of Mean Opinion Scores in Subjective Evaluation of Synthetic Speech Based on Tail Probabilities
- A statistical significance test for person authentication
- Testing Statistical Significance of the Area under a Receiving Operating Characteristics Curve for Repeated Measures Design with Bootstrapping
- Online Bootstrap Inference For Policy Evaluation In Reinforcement Learning
- Sample Average Approximation for Stochastic Optimization with Dependent Data: Performance Guarantees and Tractability
- PAC-Bayes control: learning policies that provably generalize to novel environments
- Provably Safe Sim-to-Real Transfer
- High-Confidence Policy Improvement from Human Feedback
- Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies and Conformal Prediction
- Safety Gymnasium: A Unified Safe Reinforcement Learning Benchmark
- Learning with Square Loss: Localization through Offset Rademacher Complexity