Robots trained in simulation often adapt to the real world by reading their own recent history: if a joint accelerates less than commanded, the payload is probably heavier. This works well when sensor readings arrive on a clean, fixed clock. On real hardware they do not. Packets drop, threads jitter, and the deployed controller often runs at a different rate than the simulator did. Today's history encoders, which compress the last few hundred milliseconds of motion into a small "context" vector describing the physics, are discrete networks such as 1D convolutions or LSTMs. A discrete encoder treats a missing or late sample as a different trajectory. Its estimate of the payload can therefore change because the clock changed, not because the payload did.
We propose to replace the discrete history encoder in the standard two-phase sim-to-real pipeline with a causal neural controlled differential equation. This is a continuous-time recurrent model that integrates its hidden state against an interpolated path built from the observations, so its output depends on the underlying motion rather than on where the samples happened to fall. The claim is narrow and falsifiable. Such an encoder should give stable physics estimates, and therefore stable control, under sampling regimes outside anything seen in training. Discrete encoders, even ones given explicit time-gap inputs and trained with timing randomization, should degrade outside their training range. The context update runs in a slow outer loop, which leaves time for the extra solver cost. The fast policy stays a plain feedforward network.
Core Claim
The closest existing system is RMA. It first trains a policy in randomized simulation, conditioned on a latent "extrinsics" vector computed from privileged physics parameters. It then trains a 1D-CNN to predict that vector from proprioceptive history. The adaptation module runs at 10 Hz and the policy at 100 Hz. Can Context Bridge the Reality Gap? compares feedforward and LSTM context estimators under different supervision signals. It finds that context-aware policies beat context-agnostic randomization on a real Franka pushing task, but remain below an oracle given true parameters. CaDeLaC uses an LSTM context encoder inside physics-structured MPC. All three use discrete encoders that assume a uniform clock.
On the continuous-time side, neural CDEs (Kidger et al.) and their causal online variant (Morrill et al.) handle irregular, missing data well. On CharacterTrajectories with 70% of points dropped, they reach 98.6% accuracy against 91.9% for GRU-D. However, they have been used only for passive prediction on medical and audio series. Continuous-time MBRL and the semi-MDP Neural ODE work show that continuous-time dynamics models stay stable under irregular or changed time steps where discrete models break down. Neither has a context variable, and neither does transfer across randomized physics. Neural ODE Processes infer a latent that sets an ODE's vector field from a context set. They model passive dynamics without actions, and their encoder treats observations as an unordered set.
What is new here is the specific pairing of a causal CDE with the privileged-latent distillation step of RMA. We do not propose a continuous-time world model or continuous-time RL, which would face the known cost of backpropagating through solvers. The CDE replaces only the student encoder, and it is tested where its inductive bias should matter: timing distributions the simulator never produced.
Concretely, from timestamped proprioception and commands we build a causal path $X(t) = (t, s(t), a(t))$ using rectilinear or backward-difference Hermite interpolation, both of which use no future samples. The hidden state evolves as
$$z_t = z_{t_0} + \int_{t_0}^{t} f_\theta(z_u)\, dX_u ,$$
where $f_\theta$ is a learned matrix-valued network. The context estimate $\hat c_t = g_\theta(z_t)$ is regressed onto the teacher's extrinsics $\mu(e)$, where $e$ are the true simulator parameters. Time appears as a path channel because mass and damping relate force to rates of change. The encoder must therefore stay sensitive to real time while being insensitive to how that time was sampled.
Research Program
The teacher stage follows RMA. A PPO policy conditioned on $\mu(e)$ is trained under dynamics randomization of mass, friction, damping, and motor strength, in the style of Peng et al.. The student stage trains every candidate encoder on the same rollouts with the same regression target. This isolates the encoder as the only variable. Each rollout is logged at high rate. We then resample it with a "timing randomizer" that draws a nominal rate, Gaussian jitter, and bursty packet loss from a training box, for example 50–100 Hz with up to 20% drops.
Evaluation separates three timing regimes. In-range means drawn from the training box. Shifted means a lower nominal rate of 20–30 Hz, or a higher one of 200 Hz. Adversarial means correlated drop bursts of 100–300 ms, which mimic bus stalls. Physics parameters are held fixed while timing varies, so any change in the context estimate is pure error.
We also measure what the CDE costs. Solver function evaluations and wall-clock time per context update are recorded on embedded-class CPU hardware against a 100 ms budget, which matches RMA's 10 Hz adaptation loop. A fixed-step solver variant tests whether adaptive stepping is needed at all. A secondary study tests mid-episode physics switches, such as a sudden friction drop, and records how quickly each encoder's estimate tracks the change. This addresses the static-environment assumption noted by Han & Feng. We report it but do not stake the claim on it, since discrete recurrent encoders can also track switches.
Baselines
The naive reference point is context-agnostic dynamics randomization with a feedforward policy and no history. This is the recipe of Peng et al., whose real Fetch puck-pushing success was 0.67 with randomization against 0.51 without. On real Franka pushing, Iannotta et al. report 32% success for agnostic randomization. It is trained with the same PPO budget and randomization ranges as our teacher. The upper reference is the privileged teacher itself, given true parameters at test time. All results are reported as the fraction of the agnostic-to-oracle gap closed.
The strongest existing method is RMA's 1D-CNN student and an LSTM student, both distilled from the identical teacher. The strongest reported number on this question is the 63% real Franka pushing success of the policy-loss-supervised LSTM estimator in Iannotta et al.. On simulated benchmarks we use, namely RRLS and the ODE-RL suite, the file contains no reported context-encoder results: no reported result found on Lacuna for discrete context encoders on RRLS or ODE-RL. We therefore run these baselines ourselves. To be fair, every discrete baseline receives the same timing-randomized training data. Each also gets an augmented variant that takes the elapsed time since the previous sample as an extra input, a GRU-D–style design. Each method gets the same hyperparameter search budget of 30 trials on in-range validation data and the same parameter count within 10%. Beating them means two things. First, matching them within noise in-range. Second, closing at least 15 points more of the oracle gap in the shifted and adversarial timing regimes, averaged over 10 seeds with non-overlapping 95% intervals.
The key ablation keeps everything about our encoder except the controlled integral. An ODE-RNN student evolves its hidden state by an ODE between samples and jumps at each observation, as in the semi-MDP Neural ODE work. If it matches the CDE, the benefit comes from continuous-time evolution generally, not from integrating against the path. A second ablation drops the time channel from $X$. This should hurt, because it makes the encoder blind to rates. If it does not hurt, our account of what the encoder uses is wrong. A third compares rectilinear and backward-Hermite interpolation, which trade smoothness against solver calls (Morrill et al.).
Why not just…?
Why not just randomize timing in simulation and keep the LSTM? Peng et al. already randomized timesteps and observation noise and reached 0.89 ± 0.06 on a real robot with an LSTM. This is the main threat. Within the training timing box, we expect a timing-randomized LSTM to match the CDE, and we will report that honestly. Our claim concerns extrapolation beyond the box. The shifted and adversarial regimes test exactly this. If the timing-randomized, time-gap-augmented LSTM degrades no more than the CDE there, the proposal's central claim is refuted.
Why not just resample observations onto a fixed grid and feed the existing 1D-CNN? This is cheap and plausible, and it is closely related to what the CDE does internally, since both rely on interpolation. The difference is that a CNN trained on clean grids sees interpolation artifacts from long drop bursts as unfamiliar inputs. The CDE integrates against the same interpolated path, and its training regression covers whatever artifacts the interpolation produces. We include "interpolate-then-CNN" as a baseline, using the same causal interpolation as our encoder. If it matches the CDE, the right conclusion is that interpolation, not the CDE, does the work, and we would recommend the simpler system.
Why not use classical online system identification, such as an extended Kalman filter? Kalman filters handle irregular timing natively, which makes them a fair competitor on exactly this axis. The evidence in CaDeLaC shows an EKF reducing real Franka tracking error by 21%, against 39% for the learned context model. EKFs also need an analytic model and persistent excitation. We include an EKF on the Pendulum and Franka-pushing settings, where analytic models exist, as the timing-robust classical reference.
Proposed Experiments
The simulation study uses three environments. The first is RRLS (code: SuReLI/RRLS) with the Ant, HalfCheetah, and Walker2d mass/friction uncertainty sets, using its domain-randomization wrapper. The second is the ODE-RL suite of Continuous-time MBRL, whose irregular-sampling modes give principled timing distributions. The third is CARL Pendulum, as used by Iannotta et al.. Metrics are return as a fraction of the oracle gap closed, context-estimate error against the teacher latent, and a consistency score: the variance of $\hat c_t$ across 20 resamplings of the same underlying trajectory. Latency and solver evaluations per update are also reported.
The first decisive check is the consistency score under shifted timing. The CDE's variance should stay near its in-range level, while discrete encoders' variance should grow with the distance from the training box. If both curves look alike, the method has no reason to exist. The second check is the ODE-RNN ablation, which shows whether the controlled path is the active ingredient. The third check is the latency budget. If the CDE cannot finish a context update well within 100 ms on embedded hardware with a fixed-step solver, it is impractical regardless of accuracy.
If the simulation results hold, a hardware check would repeat the Franka box-pushing protocol of Iannotta et al., with boxes of 283–824 g. The teacher would be trained in their simulator settings. Deployment would run at the nominal rate and again with artificially throttled and dropped sensor streams. The target is to match their 63% at nominal timing and to lose substantially less than the LSTM under throttling.
References
- RMA: Rapid Motor Adaptation for Legged Robots
- Can context bridge the reality gap? Sim-to-real transfer of context-aware policies
- Context-Aware Deep Lagrangian Networks for Model Predictive Control
- Neural Controlled Differential Equations for Irregular Time Series
- Neural Controlled Differential Equations for Online Prediction Tasks
- Continuous-time Model-based Reinforcement Learning
- Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs
- Neural ODE Processes
- Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
- Transferable Reinforcement Learning via Probabilistic Latent Embeddings and Dynamic Policy Adaptation for Sim-to-Real Deployment
- Rrls: Robust reinforcement learning suite