Connected vehicles can see around corners by sharing what their sensors detect with each other and with roadside cameras. That information always arrives late, often by 100 to 500 milliseconds. During that delay the car's fast controller, running at tens of hertz, still has to act. Today the late reports are brought up to the present by assuming every vehicle drove straight at constant speed. On a highway that guess is nearly right. In a roundabout, a left turn, or a curved elevated ramp, it puts reported vehicles off the road, inside medians, or below the road surface.
We propose to fix this by changing what the slow cooperative-perception layer hands to the fast control layer. Instead of a box in free space, each remote or occluded agent is passed as a position along a lane of the 3D road map, with along-lane speed and acceleration, sideways offset, and a set of possible next lanes. The fast layer moves each agent forward along the road at control rate and corrects it whenever the ego vehicle's own sensors glimpse it. The bet is that delay compensation in cooperative perception and the rate mismatch in hierarchical control are one problem with one solution. That solution matters most when driving shifts from regular highways to irregular urban and multi-level layouts.
Core Claim
Treat "vehicles stay on the road surface and follow lanes" as a contact constraint, the way physics-based pose estimation treats "feet stay on the floor." Then a cheap, causal state propagator can replace constant-velocity extrapolation in two places. The first is latency compensation inside cooperative fusion. The second is the interface between a slow planner and a fast controller.
We claim two things. This propagator should cut position error for delayed, turning, and grade-changing agents on road layouts it was never tuned on. And that gain should show up in closed loop as fewer collisions with occluded traffic, not only as higher detection scores.
Where this sits in prior work
Three cooperative-perception systems motivate the gap.
- DAIR-V2X introduced Time Compensation Late Fusion. It estimates each roadside-detected object's velocity and moves its box to the vehicle's timestamp under constant velocity. It reports up to 1.5% AP gain on asynchronous data.
- SparseCoop shares compact 11-dimensional kinematic queries (position, size, heading, velocity). It also projects them across the transmission delay with a constant-velocity model, and stays stable up to 400 ms of delay.
- DATA instead warps intermediate feature maps with learned multi-scale motion modeling.
None of the three uses road geometry when compensating delay, and all are evaluated open-loop. We keep SparseCoop's object-centric query design and replace only its motion model.
The analogy to temporal pose consistency is direct. Contact and Human Dynamics from Monocular Video shows that unconstrained frame-by-frame poses imply physically absurd ground reaction forces, above 9,000% of body weight. It also shows that enforcing contact constraints removes foot sliding and floor penetration. Its method is constrained by two limits we must avoid: it is offline (30 to 60 minutes per 2-second clip), and it assumes flat ground. The road-surface analog of foot penetration is a propagated vehicle sinking through a ramp or cutting across a median. Our constraint is enforced by parameterization rather than by optimization, so it costs microseconds per agent.
On the control side, we build on two lines of work.
- OpenCDA structures cooperative driving as separate perception, planning, cooperative-application, and actuation layers, but assumes lossless, delay-free communication.
- SuSIE shows that a slow subgoal generator paired with a fast goal-conditioned policy beats monolithic models. On CALVIN it completes 2.69 consecutive instructions versus 1.43 for AugLC. It also reports that the slow layer can emit subgoals the fast layer cannot execute.
Our handoff targets a different failure of the same split. The fast layer is handed a world snapshot that is already stale and that it cannot advance on its own.
Dual MPPI merging supplies the planner we stress-test. It is a sampling-based controller that actively probes human drivers whose behavior is modeled by a merge-specific car-following model. It assumes full visibility and a single planar ramp.
The handoff representation
Each agent $i$ is represented as $x_i = (\ell, s, d, v, a)$. Here $\ell$ is a lane in the map's lane graph, $s$ is distance along that lane's 3D centerline $\gamma_\ell$, $d$ is the sideways offset, $v$ is along-lane speed, and $a$ is along-lane acceleration. World position is recovered as
$$ p_i = \gamma_\ell(s) + d\, n_\ell(s), $$
where $n_\ell(s)$ is the in-surface unit normal to the centerline at $s$. Because $\gamma_\ell$ is a 3D curve, this automatically follows banking, ramps, and overpass decks.
Propagation over a delay $\tau$ advances $s$ by $v\tau + \tfrac12 a\tau^2$. When $s$ passes the end of a lane, the state branches into one weighted hypothesis per successor lane. Branch weights come from the agent's turn signal or heading change when observed, and are uniform otherwise.
Uncertainty is carried as a covariance in $(s, d, v)$. It grows with time since the agent was last directly observed by any agent. The fast controller receives these hypotheses as occupancy samples. A cooperative report that arrives late is folded in with a standard filter update after the report is propagated to the current time. Later ego glimpses refine it at control rate.
This is a map-constrained multiple-hypothesis filter. We do not claim the filter itself is new. The claim under test is that using it as the shared operator for cooperative delay compensation and for the planner–controller handoff closes a failure that constant-velocity propagation creates under topology shift.
Research Program
We first build the propagator into a SparseCoop-style query pipeline. Delayed queries from roadside units and other vehicles are converted to lane coordinates on arrival. They are propagated to the receiver's time, then passed to SparseCoop's coarse-to-fine aggregation unchanged. Nothing else in the detector changes, so any difference is attributable to the motion model.
Second, we wire the same agent set into a two-rate stack built on OpenCDA. The slow layer (cooperative fusion plus route-level planning) runs at the rate fused perception arrives. The fast layer (MPPI-style trajectory control) runs at 10 to 20 Hz. Between slow updates the fast layer advances agents with the road-constrained model.
Third, we design a topology-shift protocol. All motion-model and fusion hyperparameters are tuned only on straight and gently curved highway segments. Evaluation is on intersections, roundabouts, multi-lane merges, and curved or elevated ramps.
The method needs a lane graph with elevation. In CARLA-based data (OPV2V and OpenCDA co-simulation) the map is exact. For the real-world datasets the file does not state whether usable lane maps ship with them. If they do not, real-data experiments will either annotate lane graphs for a subset of scenes or report map-error sensitivity by perturbing simulated maps. Map dependence is the method's main practical risk, and we will measure it directly with offset and missing-lane perturbations.
Baselines
Do nothing. Use delayed cooperative detections as received, with no compensation. This sets the floor for how much delay hurts. On real data this is the asynchronous condition that DAIR-V2X's TCLF improved on by up to 1.5% AP.
Constant velocity. This is the strongest existing method on this exact question: SparseCoop's constant-velocity query propagation. It reports AP 0.530 and AMOTA 0.421 on V2X-Seq (SparseCoop). We run the identical query backbone and aggregation and swap only the propagator, with the same data, training schedule, and tuning budget.
Learned feature warping. DATA's progressive temporal alignment represents feature-level compensation. It reports 75.58% AP50 at 500 ms delay on DAIR-V2X-C (DATA). We train it on the same highway-only split, so its learned motion faces the same topology shift.
Map-free filter. This is an interacting-multiple-model filter with constant-velocity and constant-turn-rate modes, tuned with the same budget as our propagator. Its role is to isolate geometry from mere nonlinearity. No reported result was found on Lacuna for such a filter on any cooperative benchmark here.
Key ablation. We run our propagator with the 3D surface flattened (planar lane graph), and separately with lane branching disabled. This shows whether elevation, branching, or both carry the gain.
Closed loop. We compare against Dual MPPI given cooperative perception under constant-velocity compensation, and against passive certainty-equivalence MPPI. Dual MPPI's reported 67% merge success in IPG-CarMaker (Dual MPPI) was obtained under full observability on a planar ramp. It is a ceiling reference rather than a comparable number for our occluded setting. No reported result was found on Lacuna for closed-loop merging or intersection crossing with occluded traffic and realistic V2X delay.
What counts as beating them. Open loop, we need a statistically reliable reduction in propagated-position error and an AMOTA gain on the held-out topology split at 200 to 500 ms delay. We must also stay within noise of the constant-velocity baseline on highway segments. Closed loop, we need lower collision and near-miss rates with occluded agents at equal or better task success.
Why not just…?
Why not just use a better Kalman filter with turning modes? This is the most serious threat. An interacting-multiple-model filter captures constant-rate turns, and it may close much of the gap at simple intersections. It cannot represent which branch of a fork an agent will take. It also has no notion of road elevation, so it will still drift off curved ramps and through overpass decks. The map-free filter baseline exists to settle this question. If it matches us on the held-out split, the claim fails, and we will report the method as an engineering convenience rather than a finding.
Why not just use a learned motion forecaster? V2VNet already forecasts 3 seconds ahead from shared features. It cuts forecasting error by about 0.2 m and runs at roughly 9 ms latency, so its forecast could serve as the propagator. The concern is generalization. A forecaster trained on one set of layouts has no guarantee of keeping agents on an unseen layout's road, and DATA's learned temporal alignment raises the same concern. We include a V2VNet-style forecaster trained on the highway-only split as a propagator variant. If it matches ours on held-out topologies, learned forecasting is the simpler answer and we will say so.
Why not just inflate safety margins around stale detections? Conservative buffers trade collisions for freezing. In dense merges, passive MPPI variants reached only 37% and 31% success, against 67% for active probing (Dual MPPI). Constant-velocity error on curves grows with delay and turn rate, so a buffer large enough to cover it at roundabouts should stall the planner. The closed-loop experiment sweeps buffer size for the constant-velocity stack and reports the full success-versus-collision curve, so the trade-off is measured rather than assumed.
Proposed Experiments
Open-loop propagation. We use OPV2V, with the 8 CARLA towns split by topology and Culver City held out entirely; V2X-Seq for tracking; and DAIR-V2X-C for real-world detection across its 28 intersections. Delay is injected at 100 to 500 ms. Accuracy is measured with [email protected]/0.7, AMOTA, and propagated-position error broken down by agent turn rate and road grade. We add two road-contact violation metrics. The first is the fraction of propagated agents off any drivable surface or beyond a vertical tolerance from it. The second is the implied lateral acceleration, a direct analog of the impossible ground reaction forces reported in Contact and Human Dynamics from Monocular Video.
Closed loop. We use OpenCDA's CARLA–SUMO co-simulation with realistic latency and packet loss replacing its lossless channel. Scenarios include occluding trucks at unsignalized intersections, a roundabout entry, and an on-ramp merge with occluded mainline traffic, all fed by roadside-unit and vehicle reports. Outcomes are collision rate, near-miss rate (minimum time gap below 0.6 s, the threshold OpenCDA uses for safe merging), task success, and maneuver time.
Decisive checks. Four results decide the outcome.
- On turning and ramp agents in held-out topologies at 300 to 500 ms delay, the road-constrained propagator reduces position error against both constant velocity and the map-free filter, while matching them on highway segments.
- The flattened-surface ablation loses the ramp gain but keeps the intersection gain, which would show elevation and branching are separate mechanisms.
- In closed loop, the two-rate stack with road-constrained handoff lowers collisions with occluded agents at every buffer setting of the constant-velocity stack, without lowering success.
- Map-perturbation sweeps identify the lane-offset error at which the advantage vanishes.
If the first check fails against the map-free filter, the project stops at a negative result on whether road structure matters for cooperative delay compensation. That result is still useful, because the field currently assumes constant velocity without testing alternatives.
References
- DAIR-V2X: A Large-Scale Dataset for Vehicle-Infrastructure Cooperative 3D Object Detection
- SparseCoop: Cooperative Perception with Kinematic-Grounded Queries
- DATA: Domain-And-Time Alignment for High-Quality Feature Fusion in Collaborative Perception
- Contact and Human Dynamics from Monocular Video
- OpenCDA: An Open Cooperative Driving Automation Framework Integrated with Co-Simulation
- Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models
- Active Learning with Dual Model Predictive Path-Integral Control for Interaction-Aware Autonomous Highway On-ramp Merging
- V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction