Robot policies trained in a simulator often fail on hardware for a reason that has little to do with how the world looks. The real robot sees its sensors late and at uneven intervals. Its motors respond sluggishly, and its commands arrive after the state they were computed for has already changed. A simulator steps in clean, synchronous ticks, so a policy trained there acts on a picture of the world that on hardware is already stale. We propose to fix this at deployment time without retraining the policy. We fit a small neural ordinary differential equation (a neural network that predicts the rate of change of a system's state, which a numerical solver then integrates to produce the state at any moment) as a correction on top of the simulator's own dynamics. The fit uses a few minutes of irregularly timestamped logs from the real robot. At run time, this corrected continuous-time model takes each late, unevenly spaced measurement and rolls it forward to the moment the next command will actually take effect. The frozen simulator-trained policy is then given that state.
The motivating failure is concrete. In Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation, a policy trained in a fast rigid-body tactile simulator transferred zero-shot on four of five tasks once real tactile images were translated into simulated depth maps. It failed on the one agile task, Object Balancing, and the authors attribute the failure to robot latency and joint limits, not to the images. That is a timing gap, and continuous-time models are the natural tool for timing. Nobody has yet used them this way on hardware.
Core Claim
A residual vector field learned in continuous time will close timing-driven sim-to-real gaps that image translation leaves open. A residual vector field is a learned correction added to the simulator's rate-of-change function. The key property is that the correction is defined for every elapsed interval. It can therefore be fitted on hardware logs at one control rate and one latency profile, then used at different rates and under different jitter without refitting.
This transfer across timing conditions is the specific property that a discrete correction model cannot offer by construction. It is the property our experiments are built to test. If a discrete model that is simply told the time gap matches the continuous one on timing conditions it never saw, the claim is false.
Concretely, let $x$ be the low-dimensional state of the system: robot joint positions plus the contact pose estimated from the translated tactile image. Let $u$ be the commanded action and $z$ a small latent actuator state that captures motor lag. The model is
$$\frac{d}{dt}\begin{bmatrix} x \\ z \end{bmatrix} = \begin{bmatrix} f_{\text{sim}}(x, z) + g_\phi(x, z, u) \\ h_\phi(z, u) \end{bmatrix}.$$
Here $f_{\text{sim}}$ is the simulator's known dynamics, in which the actuator state $z$ drives the joints. $g_\phi$ is a small network that learns what the simulator gets wrong. $h_\phi$ learns how the actuators actually respond to commands. Measurements $y_k$ arrive at hardware timestamps $t_k$ with delays $\tau_k$.
Between arrivals the estimator integrates this equation. At each arrival it corrects its state with a learned gated update, the same pattern used for asynchronous sensor fusion in FusionMotion. Before each command it integrates forward over the predicted delay until the command takes effect. The policy itself is never touched.
Relation to Prior Work
The method comes from the continuous-time model-based reinforcement learning line. Continuous-time Model-based Reinforcement Learning learns ensembles of neural ODEs, which it calls ENODE, for Pendulum, CartPole and Acrobot. It shows that these models stay stable under irregular sampling and observation noise where discrete models break down, but it learns the dynamics from scratch and never leaves simulation. We keep its treatment of noise as measurement corruption. We differ in anchoring the model to the simulator, so that only the gap is learned, and in using the model as a deployment-time bridge on physical hardware rather than as a planner.
FusionMotion shows that ODE-based recurrent updates handle asynchronous camera and LiDAR arrivals on real driving logs. It does not model a simulator-to-hardware discrepancy. We borrow its gated update at measurement arrival.
CARE couples a continuous context variable to object dynamics to track drifting physical conditions. Its authors note that it cannot absorb explicit physics. Our design takes the opposite stance and makes the simulator's physics the backbone.
The file also supplies motivation and constraints. The theory in Provable Sim-to-real Transfer in Continuous Domain with Partial Observations says the sim-to-real gap grows with horizon and with the complexity of the simulator class, though only for linear-Gaussian systems. Latency effectively lengthens the horizon over which a stale state must be trusted. A small residual on a fixed simulator keeps the corrected model class narrow. We take this as intuition, not a guarantee. Flowing Through States warns that autonomous ODE trajectories cannot cross, which can bottleneck representations. Our model is conditioned on actions and operates on physical states rather than learned embeddings, so this constraint bites less, but we test for it.
Finally, ENODE reports that ODEs scale poorly to high-dimensional inputs. We therefore deliberately keep perception outside the ODE. The ODE runs on the low-dimensional pose that the existing image-translation pipeline already produces.
Research Program
The work has three stages.
Stage 1: build the pipeline and fit the residual. We rebuild the Tactile-Gym Object Balancing and Edge Following pipeline from Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation. This includes the PyBullet policy, the pix2pix translator, and a pose regressor on the translated depth image. We then log roughly ten minutes of the real robot under random excitation commands, recording hardware clock timestamps for both images and joint readings. The residual $g_\phi$ and actuator model $h_\phi$ are fitted by integrating the model between the logged timestamps and minimizing prediction error at those times. No resampling or interpolation of the logs is needed.
Stage 2: controlled timing gaps in the ODE-RL suite. Before touching hardware, we run a controlled study on Pendulum, CartPole and Acrobot from the ODE-RL suite. We construct "real" variants with first-order actuator lag, mass and friction mismatch, and observation latency drawn with exponential jitter. Policies are trained on the clean systems. Correction models are fitted on logs from one control rate and one latency distribution, then evaluated at other rates and latencies.
Stage 3: hardware and cost. We deploy on the physical robot and measure the estimator's wall-clock cost per control step. The integration has to fit inside the control period. If fixed-step solvers with two to four steps suffice, as the coarse-step result in Branched Neural Rough Differential Equations suggests is sometimes possible, solver overhead becomes a non-issue at these state sizes.
Baselines
Naive reference. The do-nothing reference is the published pipeline itself: translated images, pose estimate, frozen simulator policy, with no timing correction. This is the strongest existing method on this exact task. Its reported results are zero-shot transfer on "four out of five tasks", Edge Following error of about 1.5 mm, and Surface Following depth error of 0.57 mm, with Object Balancing reported only as a failure (Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation). No reported numeric success rate or balance duration was found on Lacuna for Object Balancing. Beating this baseline means sustained balancing where it fails, measured as balance duration and tilt error, with Edge Following no worse than about 1.5 mm.
Simulator-only forward prediction. This isolates the learned residual. We predict forward over the measured delay using $f_{\text{sim}}$ alone, which amounts to a classical delay compensator with a perfect-simulator assumption. If it matches our method, the learning contributes nothing, and the gap is pure delay rather than dynamics mismatch.
Discrete residual with the time gap as input. This is the key ablation. It is a GRU trained on the same logs, using the same data, parameter count and hyperparameter search budget, and it receives the elapsed interval as an input feature. It isolates continuity itself. Our method must beat it on control rates and latency distributions absent from the fitting logs. Matching it on the training rate is expected and does not count as a win.
Delay randomization. A recurrent simulator policy is retrained with actuator lag and latency randomized over a range covering the measured hardware values, using simulation compute equal to the original training.
ENODE learned from scratch. An ENODE ensemble is fitted to the same real logs with no simulator prior. This tests whether anchoring to $f_{\text{sim}}$ matters when real data are scarce.
Why not just…?
Why not just randomize latency and actuator lag in simulation? This is the most serious competitor, and it is the standard practice. Randomization makes the policy tolerate a spread of delays. It does not tell the policy what the state is now, so on an unstable task like balancing it tends to buy robustness by acting conservatively. The evidence in Close the Sim2real Gap via Physically-based Structured Light Synthetic Data Simulation cuts both ways. It shows that generic randomization underperforms structured modeling of the actual error source, with real-world IoU of 0.584 against 0.648. But that result concerns sensors, not timing. We run the randomized policy head-to-head with ours, and also combined with ours. If randomization alone balances as well, our correction is unnecessary for this task.
Why not just give a GRU the time gap? This is plausible and cheap, which is why it is the central ablation rather than a strawman. FusionMotion found that binned discrete recurrent models struggle at high query frequencies. That finding concerns forecasting, not control, so it does not settle the question here. The decisive evidence is the timing-transfer experiment below. If the GRU generalizes across rates as well as the ODE, we will report that the continuous formulation is not the active ingredient.
Why not just compensate delay with the simulator's model? Classical delay compensation is correct when the simulator's dynamics are right. The published failure blames latency together with actuator and joint limits, and a pure simulator prediction ignores motor lag. The simulator-only baseline measures exactly how much of the gap remains after pure delay compensation.
Why not use the ODE only as a training regularizer? Flowing Through States shows that smoothing latent trajectories with an ODE during training helps discrete agents and costs nothing at inference. That approach shapes representations inside the simulator, though. It never sees the real robot's clock, so it cannot correct a hardware-specific delay it has no data about. The inference cost it avoids is small here because the state is low-dimensional, and we measure that cost directly.
Decisive Experiments
Timing transfer. In the ODE-RL suite, correction models are fitted at a single control rate with a single latency distribution. They are then evaluated on held-out rates, both slower and faster, and on heavier-tailed jitter. The measures are prediction error at command-application time and closed-loop return. The hypothesis survives only if the continuous residual's advantage over the time-conditioned GRU grows as test timing departs from training timing.
Hardware result. On the physical robot, the headline measurements are Object Balancing duration and tilt error over repeated trials against all baselines, alongside Edge Following error to confirm that nothing slow-moving got worse. We also refit on logs recorded at one command rate and deploy at a different one. This is the hardware version of the timing-transfer test.
Diagnostics. We look for trajectory-crossing failures by checking whether prediction error spikes where contact switches on or off. We report estimator latency against the control period. If balancing still fails with an accurate state prediction, that points to joint limits rather than timing, and we would report the gap as physical rather than informational.
References
- Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation
- FusionMotion: Multi-Sensor Asynchronous Fusion for Continuous Occupancy Prediction via Neural-ODE
- Continuous-time Model-based Reinforcement Learning
- CARE: Modeling Interacting Dynamics Under Temporal Environmental Variation
- Provable Sim-to-real Transfer in Continuous Domain with Partial Observations
- Flowing Through States: Neural ODE Regularization for Reinforcement Learning
- Learning Manifold and Itô Dynamics with Branched Neural Rough Differential Equations
- Close the Sim2real Gap via Physically-based Structured Light Synthetic Data Simulation