Mobile robots and automated cars usually predict that other moving agents will keep their current speed and heading. This is the constant-velocity assumption. It is cheap and stable, and it is wrong whenever someone brakes, turns, or swerves. Most planners then make a discrete choice on top of that prediction: halt or go, yield or pass. Near the boundary between two such choices, a small error in an obstacle's observed heading swings the projected conflict point a long way and flips the decision. On real hardware this shows up as wobbling, freezing, or collisions. Simulators hide the problem because simulated obstacles tend to move in straight lines too, so the planner's wrong assumption is never punished during training.
We propose to keep the cheap straight-line projection as the robot's view of the world, and to train the decision policy in simulation against obstacles whose true motion is randomized over a broad family of non-linear behaviors. These behaviors include acceleration, braking, turning, stop-and-go, and reactive avoidance. The policy then learns to correct the systematic error of its own prediction model rather than trusting it. We call this motion-model randomization. It is domain randomization (training under many randomly perturbed simulator settings so that reality looks like one more sample) applied to the behavior of other agents instead of to the robot's own physics. It is paired with a hybrid action space: discrete modes with continuous parameters, executed by a smooth low-level controller. The test is whether this narrows the gap between simulated and physical performance among moving obstacles, and whether it removes the mode-flipping that discrete planners show near decision boundaries.
Core Claim
Two lines of evidence motivate the claim. Several works show that discrete, semantically meaningful decisions on top of crude kinematic projections are safer than precise geometric forecasts. P4P projects agents at constant velocity along lane centerlines and classifies each conflict as Yield or Pass. In closed-loop InterSim simulation it cuts collision rate from 12–14% to 1.39%, despite worse displacement error. The Invariant Environment Representation computes time-to-occupancy along the ego path under constant velocity, feeds it to a discrete-action Double DQN, and generalizes to 84.2% mean success across 13 unseen intersection types. Both are simulation-only, and both name constant velocity as the brittle point.
The other line shows what happens on hardware. Simulation-based RL for real-world driving found that discrete steering made a real car wobble, and that continuous actions with an $l_2$ penalty were needed. The BARN Challenge recorded the top simulated learned policy completing 2 of 9 physical trials, while a classical planner with explicit latency compensation completed 8 of 9.
Our claim is that these failures share a cause that existing randomization does not touch. Domain randomization so far perturbs the robot's own body. Sim-to-real quadruped locomotion randomizes the robot's dynamics, and Kinematic Domain Randomization randomizes its link lengths, doubling transfer reward over dynamics randomization (6215 vs. 3099). Neither varies how the obstacles move. The closest work, Dyna-LfLH, trains a navigator on hallucinated obstacles that are explicitly parameterized by an initial position and a constant velocity. It reaches 27% success in DynaBARN simulation and 50% on a Jackal among 11 moving Roombas. Our proposal differs from Dyna-LfLH in exactly the variable it holds fixed.
We predict that randomizing obstacle motion will do three things: raise physical success in dynamic scenes, sharply reduce discrete mode switches per second, and reduce freezing. The policy should learn a margin that scales with how far the constant-velocity projection is likely to be wrong in the current situation, instead of the fixed padding classical planners use.
Research Program
The simulator samples, for each obstacle and episode, a motion model from a parameterized family. In its simplest form each obstacle follows
$$\dot v(t) = a(t), \qquad \dot\psi(t) = \omega(t),$$
where $v$ is speed, $\psi$ is heading, and the acceleration $a(t)$ and turn rate $\omega(t)$ are piecewise-constant processes. Their switching rate, magnitude, and correlation are drawn from a distribution $p(\theta)$. Setting $a=\omega=0$ recovers the constant-velocity case, so straight-line obstacles remain a special case of training rather than being excluded. On top of this kinematic layer we mix in three further behaviors: reactive agents driven by a social-force model with randomized gains (the pedestrian model used in SOCIALGYM, but with randomized rather than fixed parameters), stop-and-go agents, and agents that reverse or redirect on contact.
The policy's input deliberately stays cheap and fixed. It consists of time-to-occupancy and time-to-vacancy features along the planned path, computed under constant velocity as in the Invariant Environment Representation, plus a short history of five LiDAR scans as in Dyna-LfLH. Because the planner's features always use the straight-line assumption while the true obstacles do not, the gap between predicted and realized occupancy becomes something the policy can learn to read from the scan history.
The action space follows the hybrid design of TS-MP-DQN, which chose discrete insertion primitives with continuous parameters and transferred zero-shot to an industrial arm at over 93% success. Here the discrete modes are Halt, Yield, Pass, and GoAlone, mirroring the SOCIALGYM sub-policies. Each mode carries a continuous target speed and lateral offset. A smooth trajectory generator executes the chosen mode, so no discrete command ever reaches the motors directly. That generator is a fitted continuous trajectory in the spirit of the Neural Implicit Action Fields argument against differencing waypoints. Training uses TS-MP-DQN, with ordinary ego-dynamics and latency randomization applied in every arm so that obstacle-motion randomization is the only variable between arms.
To avoid the over-conservatism that very broad randomization causes, we compare two ways of setting $p(\theta)$. The first is a hand-set broad range. The second is fit offline to a few minutes of tracked real obstacle motion, in the style of DROPO. Both are reported. If the fitted range matters, that is itself a finding about where the sim-to-real gap in dynamic navigation lives.
Baselines
The naive reference is the ROS Dynamic Window Approach, a local planner that samples velocity commands and scores them against obstacle positions. On the Jackal-among-Roombas setup, Dyna-LfLH reports 40% physical success for it. We run it with the same robot, sensors, and arena layouts, tuning its margins on the same simulated budget given to the learned methods.
The strongest existing learned method is Dyna-LfLH itself, at 27% success on DynaBARN simulation and 50% physical success among Roombas. It is the right competitor because it already handles moving obstacles from short LiDAR histories and already transfers to hardware, and its one structural difference from us is constant-velocity obstacles. We retrain it with its released recipe on the same compute. Beating it means a statistically clear physical success gain over 50% on the same courses, with no worse time to success. We also retrain it with our hand-set obstacle-motion family inserted into its hallucination step. If that variant closes the gap, the contribution is the randomization, not our architecture, and we will say so.
The strongest classical competitor is the dynamic control-barrier-function MPC. It tracks obstacles with a constant-velocity Kalman filter and inflates their ellipses by the filter covariance. Physically it keeps a 0.828 m minimum distance, against 0.273 m for standard MPC with barrier constraints. No success rate in dynamic scenes is reported for it on Lacuna. We run it on the same courses with its inflation rate tuned on simulated non-linear obstacles, and we report success, minimum distance, and freezing.
The key ablation is our full system trained with constant-velocity obstacles only, with identical inputs, actions, ego randomization, and training steps. It isolates motion-model randomization. A second ablation replaces hybrid actions with purely discrete actions, which isolates the chattering effect.
Why not just…?
Why not just randomize the robot's own dynamics and latency harder? Body randomization is fixed across all of our arms, so the ablation trained on constant-velocity obstacles answers this directly. Prior evidence suggests it is insufficient for structural mismatch: in Kinematic DR, dynamics randomization collapsed to a reward of 508 on the soft-floor gap, against 7305 for the structural alternative. Obstacle motion is a structural mismatch of the same kind. Friction randomization cannot teach a policy that a pedestrian might turn.
Why not just pad safety margins or inflate covariance, as classical planners do? This is a serious competitor for safety, and the barrier-function MPC baseline tests it. The expected cost is freezing. A margin large enough to cover sudden turns is too large in crowded spaces, and DWA was beaten by every finalist in BARN and by Dyna-LfLH among Roombas. We will settle it by plotting success against freezing rate as the inflation rate varies. If a single inflation setting matches our policy on both, the learned approach is not justified.
Why not just add hysteresis to the discrete switching? A dwell time or switching threshold would plausibly remove most chattering by itself, and we take that threat seriously. We add hysteresis, tuned in simulation, to the constant-velocity ablation. Our claim survives only if motion-model randomization improves success and freezing beyond what hysteresis gives. If it reduces switching but adds nothing else, we will report that the chattering half of the story was cheaply solvable.
Why not just replace constant velocity with a learned forecaster? P4P argues that precise forecasts do not buy safety: geometrically better predictors collided far more in closed loop. A forecaster trained in simulation also inherits the simulator's obstacle behavior, so it faces the same transfer problem. We nonetheless include a variant whose occupancy features come from a constant-turn-rate-and-acceleration filter, and one that uses a small forecaster trained on the same randomized data. This separates "better prediction" from "learning to distrust prediction."
Why not just use fine-grained discrete actions or fully continuous control? Discrete steering wobbled on the real car in Simulation-based RL. Fully continuous end-to-end policies are the kind that flipped in BARN. Our discrete-only ablation and a continuous PPO policy with the same inputs and randomization cover both ends.
Proposed Experiments and Decisive Checks
In simulation we evaluate in DynaBARN-style procedurally generated worlds. We hold out obstacle-motion families from training: for example, train on turn and brake processes and test on reactive social-force agents and contact-reversing agents. This measures generalization to motion the policy never saw. We also evaluate in SOCIALGYM crowds with randomized social-force gains, where the four-mode action space matches natively. As a driving-domain check, the same decision head is evaluated on InterSim interactive scenarios against P4P's 1.39% collision rate, using constant-velocity conflict features as P4P does.
Metrics are success rate, time to success, collision rate, freezing rate (the fraction of time stationary with a reachable goal), discrete mode switches per second, and peak jerk.
Physically, we use a Clearpath Jackal among 11 moving Roombas, matching Dyna-LfLH's setup, with three course layouts and at least 30 trials per method. A second scene adds walking pedestrians who are instructed to turn and stop at scripted points, giving controlled non-linear motion.
The decisive check is the simulation-to-real drop for each method. Dyna-LfLH went from 27% in simulation to 50% physically on an easier physical scene, so we report both numbers on matched layouts rather than raw transfer ratios. The proposal succeeds if three things hold. First, the motion-randomized policy beats both the constant-velocity ablation and retrained Dyna-LfLH on physical success. Second, it cuts mode switches relative to the constant-velocity-plus-hysteresis variant. Third, it does so without a higher freezing rate than tuned covariance inflation.
The proposal fails informatively in two cases. If the fitted motion distribution is necessary and the broad one hurts, randomization range rather than randomization itself is the lever. If the constant-velocity ablation with hysteresis matches everything, the straight-line assumption was never the bottleneck in these scenes.
References
- P4P: Conflict-Aware Motion Prediction for Planning in Autonomous Driving
- Generalizing Decision Making for Automated Driving with an Invariant Environment Representation using Deep Reinforcement Learning
- Simulation-based reinforcement learning for real-world autonomous driving
- Autonomous Ground Navigation in Highly Constrained Spaces: Lessons learned from The BARN Challenge at ICRA 2022
- Sim-to-Real: Learning Agile Locomotion For Quadruped Robots
- Policy Transfer via Kinematic Domain Randomization and Adaptation
- Dyna-LfLH: Learning Agile Navigation in Dynamic Environments from Learned Hallucination
- SocialGym: A Framework for Benchmarking Social Robot Navigation
- Learning Insertion Primitives with Discrete-Continuous Hybrid Action Space for Robotic Assembly Tasks
- Neural Implicit Action Fields: From Discrete Waypoints to Continuous Functions for Vision-Language-Action Models
- DROPO: Sim-to-real transfer with offline domain randomization
- Dynamic Control Barrier Function-based Model Predictive Control to Safety-Critical Obstacle-Avoidance of Mobile Robot