Many robots are now deployed with an ensemble of dynamics models (several learned predictors of the next state) trained in randomized simulators. On the real robot, the models' disagreement serves as a safety signal, and in principle their relative accuracy could be used to reweight them toward the simulator settings that best match reality. This note argues that both uses share a failure that nobody has examined in closed loop. Real sensors produce variation in input directions that the simulator never exercised. In those directions every ensemble member's behavior is unconstrained, so the members disagree for reasons unrelated to physics. That spurious disagreement makes safety-penalized policies freeze. It also corrupts the evidence used to reweight the models, so the weights end up tracking sensor artifacts rather than dynamics.
The proposed fix has two parts. First, real observations are split into the subspace the simulator actually covered and an orthogonal remainder, and only the covered part is used to score and compare models. Second, the standard Bayesian reweighting is replaced by a tempered update that includes an explicit "none of these models" member, because the real world usually lies outside the simulated family. The result is meant to be a drop-in change to disagreement-penalized sim-to-real controllers. It should keep their zero-shot safety while recovering task performance once the real robot's sensors become noisy in structured ways.
Core Claim
The claim transfers a supervised-learning result into robot control. Dangers of Bayesian Model Averaging under Covariate Shift showed that high-fidelity Bayesian model averaging (averaging predictions over many plausible weight settings) falls behind a single point estimate by up to 25% on corrupted CIFAR-10 images. The cause is weight directions attached to input features that never varied in training, such as "dead pixels." The data leave those weights free, so they are drawn from the prior and inject noise once those inputs start to vary. That paper's remedy was a prior built from the training data's empirical covariance (EmpCov). It was confined to static image classification.
We claim the same mechanism operates in sim-to-real dynamics ensembles, and that it acts through two channels. The first is the disagreement penalty of SPiDR, which adds a constant $\lambda$ times ensemble next-state variance to the safety cost. The second is any online likelihood reweighting of ensemble members, in the style of the Bayesian model combination of Reinforcement Learning with Multiple Experts.
The new element is not EmpCov itself. It is two things. One is the diagnosis that simulator-coverage gaps in sensor space contaminate both safety disagreement and model weighting on hardware. The other is a coverage-aware scoring rule that fixes both while also handling the setting where none of the candidate models is correct.
Research Program
We first build the ensemble the way SPiDR does. We train $K$ dynamics models on domain-randomized simulation, and we add members trained under alternative contact and actuator-latency approximations, so the family spans more than one parameter box.
From the simulator rollouts we compute the empirical state-observation covariance $\Sigma_{\text{sim}}$. Let $P$ be the projection onto its eigenvectors with non-negligible variance. For a real observation $o$, the covered part is $Po$ and the residual is $(I-P)o$. We fit the residual's distribution separately, from short stationary and slow-motion recordings on the hardware. The residual is treated as a sensor channel, not as evidence about dynamics.
Ensemble disagreement $\upsilon(s,a)$ and each member's predictive likelihood are computed only on the projected coordinates, and members also receive projected inputs. On real transitions, the weights $w_k$ are updated by
$$ w_k^{(t+1)} \propto w_k^{(t)}\, p_k\!\left(P o_{t+1} \mid P o_t, a_t\right)^{\beta}, \qquad k = 0, 1, \dots, K. $$
Here $p_k$ is member $k$'s predictive density, and $\beta \in (0,1]$ is a tempering exponent. We choose $\beta$ by prequential log score, which means each transition is predicted before it is used to update. Member $k=0$ is a deliberately broad model: the nominal simulator with inflated noise.
Tempering and the broad member address the known failure of averaging when every candidate is wrong. In that case, standard posterior weights collapse onto the member closest in KL divergence instead of reporting misfit. When member 0 gains weight, the controller learns that reality is outside the simulated support. It then reverts to the full SPiDR-style unweighted pessimism rather than trusting a confidently wrong member. Otherwise, the safety penalty uses weighted variance over the projected coordinates.
This design responds directly to four pieces of prior work. Gap-agnostic disagreement (SPiDR) motivates the proposal: it is the strongest existing safety method and the one we expect to over-penalize. Bayesian model combination supplies the reweighting mechanism. It learned one global weight vector over expert potential functions in Gridworld and CartPole, and assumed clean simulated observations. Shields constrains the design: overconfident misspecified priors dominate small-sample posteriors. That result is why we temper the update and keep a broad member rather than trusting early weights. SIM-FSVGD models the sim-to-real residual with a Gaussian process around a single simulator prior. We instead weight several disparate simulators and separate sensor-coverage gaps from dynamics gaps, which SIM-FSVGD does not distinguish.
Baselines
The naive reference point is plain domain randomization. It is one policy trained across randomized physics with zero-mean Gaussian sensor noise and no runtime use of ensemble information. On the Unitree Go1 it produced 26.1x more safety violations than SPiDR, and on the 1/10-scale RaceCar 6.2x more, as reported by SPiDR. We include it to anchor the scale.
The strongest existing competitor is SPiDR itself, which reports 100% real-world constraint satisfaction zero-shot on both platforms. We cannot beat 100% on safety. Beating it therefore means three things together. Constraint satisfaction must stay at or near 100%. Task return or progress must improve, or time-to-goal and freezing rate must drop. And the advantage must widen as we add structured sensor corruption that SPiDR's training did not model. SPiDR will be run with the authors' ensemble size ($n \ge 8$, where they report negligible training-time overhead), the same simulator training budget, and the same $\lambda$ tuning sweep we give our method.
The second strong competitor is BayRnTune. It fine-tunes policies while Bayesian optimization tunes the randomization distribution, and reports 25% to 400% reward gains over vanilla domain randomization in IsaacGym. We run it with the same number of real rollouts our method uses for weight updates.
The third is SIM-FSVGD, which converged in about 10 episodes on RC-car drift parking. We compare against it on next-state negative log-likelihood under matched real-data budgets.
Two ablations isolate the ingredients. The first keeps the tempered reweighting with member 0 but scores on raw observations, removing the projection. The second keeps the projection but uses untempered posterior weights without member 0. A third control applies the projection to SPiDR alone, with no reweighting, to test whether coverage-aware disagreement is the whole story.
Why not just…?
Why not just randomize sensor noise more aggressively in simulation? This is the most serious threat. If the simulator exercised every sensor direction, no directions would be left unconstrained and the projection would do nothing. Our counter is that standard randomization injects stationary, isotropic noise. Real corruption is structured and state-dependent, for example camera blur that scales with angular velocity, or LiDAR dropouts on reflective surfaces as reported by Pokle et al.. We will test this directly. A "sensor-noise randomization" baseline adds isotropic noise to all channels and is evaluated on both isotropic and structured real-style corruption. If it matches our method on the structured case, we will conclude that the projection is unnecessary and say so.
Why not just use an unweighted deep ensemble or a single regularized model? In Izmailov et al., deep ensembles and point estimates were robust to corruption. However, robust per-member predictions do not stop members from disagreeing with one another along uncovered directions, and that disagreement is what SPiDR penalizes. An unweighted ensemble also cannot shift toward the simulator variant that matches the hardware. The no-reweighting projection control above shows how much of the gain each piece provides.
Why not just run classical system identification and use one calibrated model? Bayesian Domain Randomization found that trajectory-matching identification (SimOpt) drifted to non-physical parameters on the Barrett WAM when no exact match existed. That is precisely the "outside the family" regime our member 0 is designed to flag. We include an identification-then-point-model baseline on the Furuta pendulum, where identification is most likely to succeed. If it ties there, that is useful information about where weighting is unnecessary.
Proposed Experiments
The program has three tiers.
The first tier is sim-to-sim and inexpensive. The "real" system is a different physics engine or contact model from the training simulator, with injected structured corruption: velocity-proportional blur on depth or state channels, dropouts, and slow drift. Its purpose is to verify the mechanism. The decisive check is whether raw-observation disagreement rises with corruption even when the dynamics are held identical to training, while projected disagreement stays flat. If raw disagreement does not rise, the core claim is false.
The second tier uses the continuous-control settings from Soft-Robust Actor-Critic (Cart-Pole with varying pole length, Pendulum with varying mass). These settings let us push the true parameter outside the simulated range. They test whether member 0 gains weight and whether untempered weights collapse confidently onto a wrong member. We measure weight calibration as the prequential log score of the mixture.
The third tier is hardware: the RaceCar and Unitree Go1 used by SPiDR, plus the Furuta pendulum used by BayRn. We report constraint violations, task return, freezing rate, and real-rollout count, each with and without added physical sensor degradation (taped or defocused cameras, perturbed IMU mounting).
The result that would sink the proposal is SPiDR's freezing rate not increasing under structured sensor degradation. The result that would confirm it is our method matching SPiDR's constraint satisfaction while beating it on return in exactly the corrupted conditions, with the raw-scoring ablation losing most of that advantage.
References
- Dangers of Bayesian Model Averaging under Covariate Shift
- SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer
- Reinforcement Learning with Multiple Experts: A Bayesian Model Combination Approach
- The effect of prior probabilities on quantification and propagation of imprecise probabilities resulting from small datasets
- Bridging the Sim-to-Real Gap with Bayesian Inference
- BayRnTune: Adaptive Bayesian Domain Randomization via Strategic Fine-tuning
- Deep Local Trajectory Replanning and Control for Robot Navigation
- Data-Efficient Domain Randomization With Bayesian Optimization
- Soft-Robust Actor-Critic Policy-Gradient