Motion forecasters for self-driving cars look accurate on the cities they were trained on and fail on new road shapes. Placed on realistically perturbed roads, a strong lane-graph model predicts paths that leave the drivable area 66% of the time, while a hand-written controller that tracks the road never does (Vehicle trajectory prediction works, but not everywhere). A separate line of work makes predicted paths physically drivable. The network outputs steering and acceleration commands, and a vehicle motion model turns those commands into positions (Deep Kinematic Models). Drivability alone does not keep a car on a road it has never seen, though. A perfectly drivable path can still sail straight off a curve.
We propose a recurrent forecaster in which the road's curvature enters the physics rather than the network's inputs. A recurrent network, which carries a memory of the driver's history from step to step, emits small bounded corrections at each future step: a steering correction on top of the steering needed to follow the lane, and a longitudinal acceleration. A differentiable vehicle model then integrates these commands in road-relative coordinates, meaning distance along the lane and offset from its centerline, reading the curvature of the candidate lane at the vehicle's current position. Under this design, a curve the network never saw changes the integrator's input, and the network does not need to have memorized it. The bet is that this one change produces forecasts that are both drivable and on-road under geometric shift. Kinematic layers alone and road-frame coordinates alone each deliver only one of those two properties.
Core Claim
Under shift in road geometry, two things fail separately in neural trajectory predictors. One is road adherence: the path leaves the drivable area. The other is kinematic feasibility: the path demands steering or lateral acceleration a car cannot produce. Existing fixes address one at a time.
A kinematic output layer guarantees feasibility, with 0% infeasible trajectories compared with up to 30% for unconstrained models in Deep Kinematic Models. It integrates in a fixed global frame, however, so the network itself must learn to steer through every curvature. Expressing an unconstrained model's outputs relative to the lane would keep paths near the centerline. Nothing in such a model, though, stops it from taking a tight curve at highway speed.
Our claim is that integrating a kinematic bicycle model in the lane's own coordinate frame, with lane-following steering supplied analytically, makes three things true at once: feasibility holds by construction, road adherence is the default behavior, and the speed–curvature trade-off is forced on the network. We expect this to beat both single-fix designs on the joint failure rate under the adversarial and cross-domain shifts described below, at matched in-distribution accuracy.
Model
For one candidate lane path, we use the following quantities. $s$ is the distance travelled along the lane. $d$ is the lateral offset from the centerline. $\psi$ is the heading error relative to the lane. $v$ is speed. $\kappa(s)$ is the lane curvature at the current position, and $L$ is the wheelbase. The vehicle evolves by the standard road-frame bicycle equations
$$\dot s = \frac{v\cos\psi}{1-d\,\kappa(s)},\qquad \dot d = v\sin\psi,\qquad \dot\psi = \frac{v\tan\delta}{L}-\kappa(s)\,\dot s,\qquad \dot v = a.$$
At each future step, a gated recurrent unit with hidden state $h_t$ outputs the acceleration $a_t$ and a residual steering angle. The applied steering is
$$\delta_t = \arctan\!\big(L\,\kappa(s_t)\big) + \delta^{\max}\tanh\!\big(r_t\big),$$
where $\arctan(L\kappa)$ is the steering that exactly follows the lane, $r_t$ is the network's raw residual, and $\delta^{\max}$ bounds the correction. Acceleration is bounded the same way. We also clip lateral acceleration $v^2\kappa$ at a friction limit, so the integrator decelerates the car if the network asks for an infeasible speed on a curve.
Multiple futures come from two sources. Discrete route choice comes from candidate centerlines drawn from the lane graph, with one recurrent rollout per candidate. Within-route variation comes from a small latent variable. A learned scorer ranks the modes. Initial states $(v,\psi,d)$ are estimated by smoothing the observed history, because noisy tracks would otherwise inject spurious heading errors.
This design reuses the hard kinematic layer of Deep Kinematic Models and moves it inside a recurrent loop, which that paper left as future work. The lane-relative frame and the analytic feedforward steering are the new ingredients.
Relation to Prior Work
Several papers motivate this design. Vehicle trajectory prediction works, but not everywhere builds adversarial road scenes using smooth-turn, double-turn, and ripple-road edits, filtered so that each curve is drivable at a physically achievable speed. It finds that LaneGCN's hard off-road rate climbs from near 0% to 66% on these scenes, and that fine-tuning on them only reduces it to 46%. The paper repairs the data. We change the model's structure instead.
TransferTraj obtains large zero-shot gains across cities by encoding only relative displacements. It operates on macro-scale GPS sequences, however, and imposes no vehicle limits. Our road frame plays a similar coordinate-removing role for micro-scale motion, but it is tied to the physics.
Other papers constrain the design. PITA reports that a strict action-space autoencoder reconstructs real rounD trajectories poorly. Its authors therefore fall back on soft physics penalties with an exponentially increasing weight, which prevents the network from collapsing to stationary trajectories.
This tension between hard and soft constraints is the main risk to our proposal. Our hypothesis is that hard rollouts lose accuracy mainly because the network must produce large, noise-sensitive steering commands. When lane-following steering is supplied analytically, the residual it must learn is small, and the accuracy penalty should shrink. Our experiments test this directly rather than assume it.
Goal-based Neural Physics uses LSTMs to set the parameters of a point-mass force model on highways, reaching an RMSE of 1.59 at 5 s on NGSIM. A point mass can translate sideways, so that model does not ensure the non-holonomic feasibility that our bicycle model enforces.
The microfluidics surrogate in Ensemble latent assimilation shows that an unconstrained LSTM rollout drifts, and that external correction from observations is needed to stop the drift. Here, the drift is meant to be controlled internally, since heading error is measured against the road and does not accumulate in a global frame. Physics-Informed Trajectory Prediction under Missing Observation uses a bicycle model only as a filter on the history, not on the forecast.
Research Program
We will build the road-frame kinematic recurrent forecaster on top of a standard lane-graph context encoder, so that the only change is the decoder. We will then construct a controlled series of decoders that share this encoder, recurrent core, and multi-modal head. They differ only in two respects: whether a kinematic integrator is present, and whether the decoder operates in a global frame or in the road frame.
The analysis has three parts. The first separates off-road failures from infeasibility failures under shift. The second measures how the hard-versus-soft accuracy gap from PITA changes once lane-following steering is supplied analytically. The third checks whether long-horizon drift in heading and speed is reduced in the recurrent rollout. We will also report how often the friction clip activates. Frequent clipping would mean the network relies on the integrator to fix speeds it gets wrong.
Baselines
The naive reference points are two physics-only predictors with no learning. The first is constant velocity and constant yaw rate. The second is the rule-based model-predictive controller from Vehicle trajectory prediction works, but not everywhere, which tracks the lane and reported a 0% off-road rate on the adversarial scenes. We run both on identical scenes.
The file contains no reported displacement accuracy for this controller. Our expectation is that it is on-road but inaccurate. If it instead matches learned models on displacement error under shift, then the learned recurrent core contributes little, and we would report that.
The strongest existing method is LaneGCN from Learning Lane Graph Representations for Motion Forecasting. It reports minFDE (K=6), the endpoint error of the best of six predicted paths, of 1.36 m and a miss rate of 0.16 on Argoverse. It also has the documented 66% hard off-road rate on adversarial scenes, and 46% after adversarial fine-tuning.
We will retrain it with the authors' recipe on the same training split, with and without the adversarial fine-tuning. Our model gets the same tuning budget in GPU-hours and the same number of hyperparameter trials. Beating it means two things together. On adversarial and rounD scenes, the joint failure rate (off-road or infeasible) must be at least halved relative to the fine-tuned LaneGCN. On in-distribution Argoverse, minFDE must stay within 5% of its reproduced value.
The kinematic-layer baseline follows Deep Kinematic Models, which reports 0% infeasible paths against up to 30% for unconstrained models. Here it is implemented as a global-frame bicycle integrator behind the same recurrent core.
Two ablations isolate our key ingredients. The first is a road-frame decoder with no integrator, which directly regresses offset along and across the lane. The second is our full model with the analytic feedforward steering term removed. Comparing the full model with the first ablation measures what the physics adds. Comparing it with the second measures whether the analytic feedforward, not just the frame change, is what preserves accuracy.
Why not just…?
Why not just put a kinematic layer on an RNN and stop there? That is the direct extension of Deep Kinematic Models, and it guarantees feasibility. Integration in a global frame, however, leaves the network responsible for producing the right steering on every unseen curvature, which is the memorization that Vehicle trajectory prediction works, but not everywhere identifies as the root failure. Our global-frame kinematic baseline tests this. We predict near-zero infeasibility but a substantial off-road rate on ripple-road and double-turn scenes. If its off-road rate matches ours, the road frame is unnecessary, and we would say so.
Why not just express an ordinary predictor's outputs in lane coordinates? This is the most serious threat. Lane coordinates alone may remove much of the off-road failure. Our first ablation is exactly this model. Our hypothesis is that it stays on the road but drives tight curves at infeasible speeds, because nothing in it ties speed to curvature. The adversarial scenes are a sharp test of this, since they were filtered precisely by the speed achievable at each curvature.
If this ablation matches our model on both off-road rate and infeasibility, the integrator does not earn its place, and the honest contribution would become the frame change.
Why not just augment training with adversarial roads? Vehicle trajectory prediction works, but not everywhere already tried this and left a 46% hard off-road rate. Augmentation also gives no guarantee on curvature families that were not generated. We include augmentation as an add-on to every model, so the experiment shows whether structural change and augmentation stack or substitute.
Why not just use soft physics penalties, as in PITA? Soft penalties leave residual violations, and they need loss scheduling to avoid collapse to stationary trajectories (PITA). We train a soft-penalty variant of the road-frame decoder under the same budget. If it reaches equal accuracy with negligible violations under shift, hard integration is not justified on accuracy grounds. Its remaining advantage would then be guaranteed feasibility.
Proposed Experiments
Training and evaluation data. Training uses Argoverse (Pittsburgh and Miami). Evaluation covers three settings. The first is the standard Argoverse validation set, which measures in-distribution accuracy. The second is the three adversarial scene families from Vehicle trajectory prediction works, but not everywhere. The third is zero-shot transfer to rounD roundabouts, where lane centerlines are taken from the map or reconstructed from aggregated trajectories, and to HighD highways.
Metrics. We report minADE and minFDE at K=6 (the average and endpoint error of the best of six predicted paths), miss rate, and hard off-road rate. We also report a kinematic infeasibility rate: the fraction of predicted paths exceeding steering, curvature-rate, or lateral-acceleration limits, computed identically for every model. Their union, the joint failure rate, is the headline metric under shift. We also report heading-error growth against prediction horizon.
There is no reported result on Lacuna for any learned model's accuracy on rounD forecasting, or for any physics-constrained model's off-road rate on the adversarial scenes. We will produce these numbers rather than cite them.
Decisive checks. There are three. First, our model must cut the joint failure rate under shift well below both the global-frame kinematic baseline and the unconstrained lane-coordinate ablation, each of which is expected to fail on only one of the two failure types. Second, in-distribution minFDE must stay within 5% of the reproduced LaneGCN value, which tests whether analytic feedforward steering removes the accuracy cost of hard rollouts reported by PITA. Third, the friction clip must activate rarely on in-distribution data, showing that the network learned sensible speeds rather than leaning on the integrator.
Failure on the first check would mean that kinematics and frame choice are interchangeable fixes. Failure on the second would mean the hard-versus-soft tension persists even when lane-following steering is supplied analytically. Either outcome is informative and would be reported as such.
References
- Vehicle trajectory prediction works, but not everywhere
- Deep Kinematic Models for Kinematically Feasible Vehicle Trajectory Predictions
- TransferTraj: A Vehicle Trajectory Learning Model for Region and Task Transferability
- PITA: Physics-Informed Trajectory Autoencoder
- Goal-based Neural Physics Vehicle Trajectory Prediction Model
- Ensemble latent assimilation with deep learning surrogate model: application to drop interaction in a microfluidics device
- Physics-Informed Trajectory Prediction for Autonomous Driving under Missing Observation
- Learning Lane Graph Representations for Motion Forecasting