A robot trained in simulation usually meets reality as a physics setting it has not seen: heavier, slipperier, or slower to respond than expected. Domain randomization is the standard remedy. It trains the controller across many randomly perturbed copies of the simulator, which makes the controller good on average but offers no guarantee of avoiding failure. Hamilton-Jacobi (HJ) reachability, a control-theory tool that computes the set of states from which no action can prevent a crash, does offer such a guarantee. It pays for it with a dilemma. If the unknown physics is treated as an adversary ranging over the whole randomization range, the robot becomes so cautious it barely moves. If the physics is averaged away, the "safe" set quietly becomes unsafe for the sluggish end of the range.
We propose a safety filter built on a physics-conditioned world model: a learned simulator that predicts the next state given the current state, the action, and an explicit physics parameter such as mass, friction, or turning authority. The filter's value function is conditioned on a credible set of physics parameters, meaning the parameters still consistent with what the robot has observed during deployment. Worst-case reasoning is applied only over that set. At the first step of deployment the filter is as cautious as a full-range worst-case method. As the observed history rules out parameters, the certified safe region grows toward what an oracle that knew the true physics would allow. When no parameter in the randomized range explains the observations, the same world model detects this and forces the robot into a backup maneuver. Everything is trained in randomized simulation, and the filter is deployed zero-shot, with no real-world fine-tuning.
Core Claim
Everything here is trained zero-shot in simulation. The central claim is that the tail of the physics distribution, which domain randomization misses and which average-case methods ignore, is better handled by a reachability game whose adversary is confined to a shrinking, data-driven credible set than by either of the usual alternatives. Those alternatives are a variance penalty added to an expected cost, or worst-case reasoning over the entire randomization range. This proposal predicts three specific outcomes. First, conditioning on the credible set should match the constraint satisfaction of a full-range worst-case filter while recovering most of the reward that filter gives up. Second, it should strictly reduce violations relative to a filter computed on a parameter-averaged world model. Third, the world model's prediction residual should give a usable alarm when reality falls outside the randomized range, which none of the zero-shot baselines provide.
The motivating evidence comes from two places. Your Learned Constraint is Secretly a Backward Reachable Tube shows that safe regions depend on the platform's dynamics. A backward reachable tube (BRT) is the set of states from which failure is unavoidable. A BRT computed for an agile platform is unsafe when applied to a sluggish one, and the reverse is needlessly conservative. A world model trained on a randomization mixture without parameter conditioning learns something close to the average platform, so the tube it produces should inherit exactly this under-conservatism at the sluggish extreme. SPiDR shows the practical cost of average-case training: standard domain randomization produced 26.1x more safety violations than SPiDR's pessimistic variant on a Unitree Go1.
Mechanism
We write $z$ for the world model's latent state, $a$ for the action, $\xi$ for the physics parameter, and $S$ for the credible set of parameters. $S$ is represented as a box in the low-dimensional parameter space. It is produced by a history encoder and calibrated with conformal prediction, a distribution-free method that turns held-out errors into sets with a guaranteed coverage rate. The world model $f(z,a,\xi)$ is trained in simulation, where $\xi$ is known for every trajectory. A failure margin $\ell(z)$ is negative inside the failure set. The safety value is trained in imagination with a discounted safety Bellman backup:
$$V(z,S) = (1-\gamma)\,\ell(z) + \gamma \min\Big\{\ell(z),\ \max_{a}\ \min_{\xi \in S} V\big(f(z,a,\xi),\, S'\big)\Big\}$$
Here $\gamma$ is the discount and $S'$ is the credible set after one more observation. At training time, $S'$ is simulated by drawing a true $\xi^\star \in S$ and updating the encoder on the imagined transition. The inner minimum is approximated by evaluating a small grid plus adversarially refined samples of $\xi$ inside the box. At deployment, a nominal policy runs freely while $\min_{\xi\in S} V(f(z,a,\xi),S) > \delta$. Otherwise the action maximizing that quantity is substituted. The margin $\delta$ is a conformal buffer on held-out randomized episodes, in the style of Safe Flow Q-Learning.
Two limiting cases fall out of the same object. Fixing $S$ to the full randomization range gives the frozen-robot worst case. Shrinking $S$ to the true $\xi$ gives the oracle. The out-of-support alarm fires when the smallest one-step prediction residual over all $\xi$ in the full range exceeds its conformal threshold. Reality is then treated as failure, which borrows the move UNISafe makes for high epistemic uncertainty.
Relation to Prior Work
Three lines of work supply parts of this design.
Latent reachability filters supply the method. Generalizing Safety Beyond Collision Avoidance via Latent-Space Reachability Analysis solves HJ reachability inside a learned visual world model and cut violations of an imitation policy by 63.6%. UNISafe adds ensemble-disagreement penalties and cut real Jenga failures from 0.82 to 0.08. Both train on real data. The first explicitly broke when bag dynamics changed, from Skittles to M&Ms. Neither conditions on physics or uses randomization. We keep their latent reachability machinery and replace fixed-data epistemic uncertainty with explicit, shrinking physics uncertainty inherited from the simulator.
Zero-shot safe transfer supplies the strongest competitors. SPiDR penalizes disagreement across randomized simulators inside an expected-cost objective and reports 100% constraint satisfaction on hardware. Transferable RL via Probabilistic Latent Embeddings infers a physics context online and relaxes risk aversion as it contracts. That second paper is the closest in spirit to our shrinking set, but it uses tail quantiles of a cumulative cost rather than a reachable set, and it assumes the context is stationary. Sim-to-Lab-to-Real does combine reachability-style shielding with randomization, but its critic is model-free and fixed after the Sim stage, and it needs an intermediate Lab stage to certify.
Expected-cost world-model safety constrains the design. SafeDreamer reaches near-zero cost in single simulators but optimizes an average, which permits rare boundary breaches. We treat it as a baseline rather than a foundation.
Research Program
The work proceeds in three stages of increasing realism.
In the first stage, we use a Dubins car whose turning authority is randomized over a range. There we can compute exact tubes for each parameter with a grid solver, as in Robust Control Barrier-Value Functions, and so measure tube error directly. The first question is whether the credible-set value converges to the per-parameter ground truth as history accumulates. The second is whether the averaged model under-covers at low turning authority, as the constraint-transfer analysis predicts.
In the second stage, we move to the MuJoCo-JAX port of Safety Gym PointGoal2 used by SPiDR, with mass and damping randomized, including the out-of-distribution settings used by Han and Feng. We also use the Hopper, Walker, and Inverted Pendulum tasks from RRLS, with falls defined as failure.
In the third stage, if hardware access allows, we deploy on the 1/10-scale RC race car setting SPiDR used, with tire friction and actuation latency randomized in simulation.
A pixel-input variant of the world model is a stretch goal. It is where the world model is indispensable, because simulator state is not available at deployment.
Baselines
The naive reference is a domain-randomized policy with no filter, trained with PPO-Lagrangian on the same parameter distribution. It establishes how many violations average-case training leaves in the tail. On PointGoal2 under out-of-distribution mass and damping, Han and Feng report a nominal cost of about 29.6 against a limit of 10.
The strongest zero-shot method is SPiDR. We run it from its published setup on the identical MJX environments and parameter ranges, give its penalty weight $\lambda$ the same hyperparameter search budget as our margin $\delta$, and match the number of environment steps. SPiDR reports 100% constraint satisfaction on both Go1 and the race car, and 6.2x fewer violations than standard randomization on the race car. On our simulated tasks we have no reported SPiDR violation numbers; no reported result was found on Lacuna for SPiDR's exact cost on PointGoal2 beyond being outperformed by Han and Feng. Han and Feng's method is the strongest reported number on PointGoal2, with a cost of 8.86 under out-of-distribution physics. We rerun it under our seeds.
Beating these methods means one of two outcomes. We could match or lower their violation rate while increasing reward. Or, more importantly, we could show zero violations on parameter settings at the sluggish edge of the range, or just beyond it, where they fail.
SafeDreamer trained under randomization, and a UNISafe-style ensemble filter trained on randomized data, test whether existing world-model safety methods already suffice once given the same data.
The key ablations isolate the credible set. The first computes the identical Bellman backup with $S$ fixed to the full range. The second uses a parameter-free averaged world model. The third gives the filter the true $\xi$, which serves as the oracle upper bound. We report what fraction of the oracle's certified safe region each variant recovers.
Why not just…?
Why not just use SPiDR, since it already achieves 100% constraint satisfaction on hardware? SPiDR is a real threat on in-support physics, and we may only tie it there. Its guarantee rests on reality lying inside the randomized support, which it explicitly assumes. Its penalty $\lambda$ also trades safety for reward with no notion of what has been learned during deployment. The decisive experiment is a sweep of test parameters from the center of the range to beyond its edge. If SPiDR stays at zero violations beyond the edge with reward comparable to ours, the out-of-support alarm adds nothing and we should say so.
Why not just run a full-range worst-case reachability filter? Its safety is not in doubt. Its reward is. The fixed-$S$ ablation is exactly this method, so the comparison is clean. The idea fails if the credible set contracts too slowly within an episode to matter, which the Dubins convergence curves will reveal early.
Why not train UNISafe's ensemble filter on randomized simulator data and let disagreement stand in for physics uncertainty? Ensemble disagreement trained on a mixture conflates "which physics" with "model error". It does not shrink as the robot observes its own dynamics, so it should behave like a fuzzy version of either the averaged or the full-range filter. This is tested directly as a baseline. If it matches our method, then explicit parameter conditioning is unnecessary.
Why not skip the world model and compute the backup with the simulator itself, branching over $\xi$ from reset states? In state-based simulation this is feasible, and it would likely match our value function. We include it as a privileged variant. The world model earns its place in two ways. It supplies the prediction residual that drives the out-of-support alarm at deployment. And it enables the pixel-input setting, where no simulator state exists. If the privileged variant equals ours and the alarm proves useless, the model-based component reduces to an engineering convenience.
Decisive Checks
The proposal stands or falls on three measurements.
The first is tube error on the Dubins car. The averaged model should show a measurable under-coverage of the true tube at low turning authority. The credible-set value should close this gap within a bounded number of observed steps.
The second is the test-parameter sweep on PointGoal2 and RRLS. Here the method should hold zero violations across the randomized range, where averaged training and SafeDreamer fail at the edges. It should also recover substantially more reward than the full-range worst case.
The third is beyond-range deployment. Here the residual alarm should fire before violations occur, at a false-alarm rate on in-range episodes bounded by its conformal calibration. If hardware is available, a race car run with deliberately degraded tires checks whether the third result survives real latency and friction.
References
- Your Learned Constraint is Secretly a Backward Reachable Tube
- SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer
- Safe Flow Q-Learning: Offline Safe Reinforcement Learning with Reachability-Based Flow Policies and Conformal Prediction
- UNISafe: Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures
- Generalizing Safety Beyond Collision Avoidance via Latent-Space Reachability Analysis
- Transferable Reinforcement Learning via Probabilistic Latent Embeddings and Dynamic Policy Adaptation for Sim-to-Real Deployment
- Sim-to-Lab-to-Real: Safe RL with Shielding and Generalization Guarantees
- SafeDreamer: Safe Reinforcement Learning with World Models
- Robust Control Barrier-Value Functions for Safety-Critical Control