Cheap robots are now good enough to run policies trained entirely in simulation. A roughly $930 radio-controlled car can learn to drift in a GPU simulator and do it on a real floor with no real-world training. These robots still fail in a predictable way. The policy was trained to cope with a fixed range of simulated physics, and once the real car moves outside that range it keeps executing as if nothing had changed. The car may have leaked outside the range because a tire wore down, the battery sagged, a payload was added, or the floor changed from carpet to tile. Expensive robots handle this with better sensors and stiffer hardware. Cheap robots cannot, and they cannot afford to rerun a heavy planner every control step either.
We propose a small onboard monitor that measures, at every step, how surprised a simulation-trained model of the robot's own motion is by what actually happened. That surprise signal drives a three-tier response. When surprise is low, the robot does nothing. When it is moderate, the robot re-estimates a handful of physical parameters, such as friction, steering offset, and actuation delay, and passes them to the policy. When the best parameter fit still cannot explain the data, the robot knows it is outside everything it was trained on, so it slows down and replans its route. The larger bet is that this monitor changes how wide the simulated physics range needs to be. With the monitor in place, a policy can be trained on a narrower, less cautious range, and it can go faster without failing more often.
Core Claim
The mismatch between simulation and reality (the "reality gap") and the question of when to replan are usually treated as separate problems. They are really one measurement. A forward model trained across randomized simulated physics predicts a distribution over the robot's next state. The error between that prediction and the observed state is the reality gap made visible online. On a low-cost platform, this error is cheap to compute. It is also the right signal for two decisions: whether to adapt the physics estimate, and whether the current plan is still valid.
Concretely, let $s_k$ be the proprioceptive state (wheel odometry, IMU yaw rate, lateral acceleration) and $a_k$ the commanded action at step $k$. Let $\mu_\theta, \Sigma_\theta$ be the mean and covariance predicted by a small forward model conditioned on a physics parameter vector $\theta$. Over a sliding window of $W$ steps we compute a surprise score
$$ z_t(\theta) = \frac{1}{W}\sum_{k=t-W}^{t-1} \big(s_{k+1}-\mu_\theta(s_k,a_k)\big)^\top \Sigma_\theta^{-1} \big(s_{k+1}-\mu_\theta(s_k,a_k)\big). $$
With the current estimate $\hat\theta$ and two thresholds $\tau_1 < \tau_2$, the rule is as follows. If $z_t(\hat\theta) < \tau_1$, the robot keeps executing. If $z_t(\hat\theta) \ge \tau_1$, it refits $\hat\theta = \arg\min_{\theta \in \Theta} z_t(\theta)$ inside the training range $\Theta$. If the refit residual still exceeds $\tau_2$, or the fit lands on the boundary of $\Theta$, the robot declares itself out of range and escalates to a route-level replan in a conservative driving mode. A budget gate limits how often each tier can fire.
What would be new is the combination of three things. The replan trigger is a physics residual rather than a plan-likelihood score. The trigger escalates by tier, from parameter refit to replanning. And we measure, on real sub-$1,000 hardware, how much variation should be absorbed passively by training-time randomization versus caught actively at runtime.
Where This Sits in Prior Work
The application regime and the strongest baseline both come from Wheeled Lab (also Wheeled Lab). That work connects the MuSHR car (about $930) and the HOUND car (about $3,000) to Isaac Lab. It achieves zero-shot drifting, elevation traversal, and visual navigation through aggressive domain randomization, meaning training across many randomly perturbed versions of the simulator. It also reports that steering inaccuracies and suspension quirks hurt transfer and were handled with hardware fixes such as taping tires. Its policies are purely reactive and have no way to notice they have left the training range. We reuse its platforms, tasks, and training pipeline, and add the monitor on top.
Two lines of work supply the idea of triggered replanning, and both constrain our design. Adaptive Online Replanning with Diffusion Models triggers replanning when a diffusion planner's own trajectory likelihood drops, with a cheap local repair and an expensive full regeneration. That two-tier structure is the one we borrow. Its signal, however, is kinematic plausibility rather than physical mismatch, and repeated denoising is costly on embedded compute. When Replanning Becomes the Bottleneck shows that unbudgeted replanning with transformer planners produces heavy-tail latency spikes and real-time deadline violations. We adopt its cooldown and commit windows as our budget gate, but our replanner is a lightweight geometric one rather than a language model. ATLAS similarly replans when observed transitions deviate from an internal look-ahead model, but it operates in web environments with no physics.
Three works motivate the physics side. DrEureka uses a language model to choose randomization ranges from stress tests of the policy, but it does all of this before deployment and names adaptive randomization as open; our monitor is the runtime half of that loop. Sim-to-Real RL for Vision-Based Dexterous Manipulation on Humanoids tunes simulator friction and damping from under four minutes of real data. That is a strong, simple answer to unit-to-unit variation, and it becomes one of our baselines. SR² and RGAT ground the simulator using real data or a learned action correction. SR² needs simulator resets to exact real states, which cheap robots without motion capture cannot provide. RGAT was never tested on hardware. Both are offline, whereas our correction is online and restricted to a low-dimensional $\theta$.
Evidence that actuation delay and inertia mismatch dominate the gap for fast platforms comes from Peng et al. and Kelchtermans et al.; this is why delay is in $\theta$. Dynamic Compliance Tuning is the closest precedent for acting on a runtime discrepancy between planned and measured physical quantities. It relies on a 6-axis force sensor, whereas we use only the odometry and IMU that a MuSHR already carries. Finally, OpenAI's in-hand manipulation work is the main counterweight: it shows that extreme randomization alone can produce robust feedback behavior. We take it seriously and test it directly rather than assuming it fails.
Research Program
We will build four components.
The first is the policy. It is a Wheeled Lab drifting and navigation policy retrained so that it takes an estimate of $\theta$ as an extra input. Here $\theta$ covers tire friction, steering offset, actuation delay, motor gain, and payload mass. We train it at three randomization widths: narrow, Wheeled Lab's width, and twice that width.
The second is a forward model. It is a small network, under 100k parameters, trained on the same simulated rollouts. It predicts a Gaussian over the next proprioceptive state given the recent state, the action, and $\theta$. It must run well within the control period on a Raspberry Pi 4 or a low-tier Jetson module. The refit is a few hundred candidate $\theta$ values scored in one batched forward pass, not gradient descent, so its cost is bounded.
The third is the out-of-range response. It combines a geometric route replanner (a shortest-path search over the known track or occupancy grid) with a conservative speed cap, and it is gated by cooldown and commit windows in the style of the budgeted replanning work.
The fourth is a perturbation protocol that makes the hardware drift deliberately and repeatably, both between trials and in the middle of a run. We will change tires or tape mid-run, add clip-on payloads, run at low battery, shim the steering servo to create a known offset, switch the floor surface along the track, and place obstacles that block the planned route. We will run the whole protocol on at least three MuSHR units, which exposes manufacturing variation from unit to unit. A second platform, HOUND, tests whether the thresholds transfer across cars.
Before touching hardware, we will run an Isaac Lab surrogate study. There, "reality" is a held-out simulator with parameters partly outside $\Theta$, following the sim-to-sim evaluation style of RGAT. This lets us set $\tau_1$ and $\tau_2$ and draw the full trade-off curves cheaply.
Baselines
The naive reference point is the nominal policy trained with narrow randomization and no monitor. It shows how large the gap actually is on these cars under our perturbations. If the full system does not clearly beat this reference, the perturbations are too weak to be informative.
The strongest existing method is Wheeled Lab's wide-randomization drifting and navigation policy, run as released on the same cars, tracks, battery schedule, and perturbation sequence. It gets the same simulator compute budget as our policies; the monitor adds nothing to training compute. Its reported drifting result is a maximum controlled slip angle of 58° at an average speed of 1.6 m/s with full lap completions (Wheeled Lab). No success rate under mid-run perturbation is reported on Lacuna, so we will measure one ourselves under our protocol.
Beating this baseline has a specific meaning. Under perturbation, our system must have a higher lap-completion rate at equal or higher average speed. Alternatively, it must match completion while running at least 15% faster. Either comparison is made with the same number of seeds and trials per condition.
The second strong competitor is per-unit calibration: before deployment, the simulator is tuned to each car from a few minutes of driving, as in the humanoid autotuning work, and then the car runs with fixed parameters. This is the best answer to unit-to-unit variation. We give it the same calibration data our refit sees at the start of each session.
The third competitor is a likelihood-triggered replanner in the style of RDM. It uses a diffusion trajectory planner distilled to fit on the Jetson and replans whenever its own trajectory likelihood drops. This tests whether a physics residual is a better trigger than plan plausibility.
Three ablations isolate the key ingredient. The first, always-on refit, re-estimates $\theta$ every step with no gate. The second, trigger-only, sends every surprise event straight to a route replan and never refits. The third, fixed-rate replanning, replans at a constant rate with no monitor at all. For each ablation we report onboard compute and latency alongside task metrics, because compute is the resource these platforms lack.
Why not just…?
Why not just widen the domain randomization? This is the real threat, since Wheeled Lab and OpenAI's hand work both succeed this way. The file points to a cost: Dynamic Compliance Tuning reports that uniformly randomized policies fail completely at tight insertion tolerances, which is consistent with wide randomization buying robustness through caution. Whether the same thing happens on a car is exactly what our width-by-monitor experiment measures. If twice-wide randomization without a monitor matches our system on completion and speed, we will report that. The claim would then shrink to the out-of-range detection that no width can provide, for example a blocked route or a surface outside $\Theta$.
Why not just calibrate each unit once before deployment? Calibration fixes offsets that stay constant, and it will probably handle unit-to-unit variation as well as we do. It cannot follow parameters that change mid-run, such as battery sag, tire wear, or a surface transition, and it cannot tell the robot when it has left its training range. The per-unit calibration baseline, split by static versus mid-run perturbations, settles this. We expect a tie on static perturbations and a win on mid-run ones. If calibration also wins mid-run, the drift we can produce is too slow to matter, and we will say so.
Why not just re-estimate the physics every step, or replan at a fixed rate? Continuous online estimation of system parameters is the obvious simpler design, and on task metrics it may match surprise-gating. The gate earns its place in two ways. First, it saves compute on a Raspberry Pi-class board, where refitting and replanning every step compete with perception for the same processor. Second, its residual-after-refit test is the only mechanism in the design that decides between refitting and replanning. The always-on and fixed-rate ablations measure both effects. If always-on refit matches task performance at acceptable latency, we will report gating as a compute optimization rather than a capability.
Why not just use a large-model replanner with a latency budget? The budgeted replanning work already makes language-model replanning workable on standard robots. The failures we target, however, are physical, not semantic. Low friction or delayed steering is visible in the IMU long before it shows up in anything a language model would reason about, and running such a model onboard at control rate is out of reach at this price point.
Proposed Experiments
The decisive experiment is a three-by-two grid: randomization width (narrow, Wheeled Lab, twice-wide) crossed with monitor (off, on). Each condition runs the drifting task and the figure-eight navigation task on three MuSHR units under the full perturbation protocol. We report lap-completion rate, average speed, collisions, time from perturbation onset to a stable lap, and replans per minute; the last measures churn, meaning the plan flipping back and forth. The hypothesis is that narrow randomization with the monitor on reaches higher speed than twice-wide randomization with the monitor off, at comparable completion. If that holds, it is direct evidence for how passive randomization and active replanning should share the work.
The second check is whether the monitor tells the tiers apart correctly. On surrogate trials we know whether true parameters were inside or outside $\Theta$, so we can compute how often "out of range" fires correctly, and how quickly, using detection precision and recall. On hardware we use the shimmed-servo and payload settings, where ground truth is known.
The third check is resource use. We report per-step latency distributions on the Pi and Jetson, including the 99th percentile, and the rate of missed control deadlines, compared against the fixed-rate and diffusion baselines. This follows the deadline-violation reporting style of the budgeted replanning work.
The final check is portability. We move the MuSHR-tuned thresholds to HOUND without retuning. If performance holds, the approach is a reusable addition to open low-cost pipelines rather than a per-robot engineering effort. We will release the monitor, the perturbation protocol, and the trial logs so other groups with $1,000 robots can reproduce the grid.
References
- Wheeled Lab: Modern Sim2Real for Low-cost, Open-source Wheeled Robotics
- Adaptive Online Replanning with Diffusion Models
- When Replanning Becomes the Bottleneck: Budgeted Replanning for Embodied Agents
- ATLAS: Actor-Critic Task-Completion with Look-ahead Action Simulation
- DrEureka: Language Model Guided Sim-To-Real Transfer
- Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids
- State Revisit and Re-explore: Bridging Sim-to-Real Gaps in Offline-and-Online Reinforcement Learning with An Imperfect Simulator
- Reinforced Grounded Action Transformation for Sim-to-Real Transfer
- Learning Agile Robotic Locomotion Skills by Imitating Animals
- RARA: Zero-shot Sim2Real Visual Navigation with Following Foreground Cues
- Bridging the Sim-to-Real Gap with Dynamic Compliance Tuning for Industrial Insertion
- Learning dexterous in-hand manipulation