TL;DR
This paper introduces a model-agnostic, late-fusion framework for collaborative vehicle-to-vehicle (V2V) trajectory prediction. By sharing sparse, asynchronous trajectory forecasts instead of heavy perception feature maps, the proposed method drastically reduces communication overhead while significantly improving prediction accuracy and safety coverage under severe occlusions.
What Problem Does This Paper Address?
Autonomous vehicles (AVs) rely heavily on predicting the future paths of nearby pedestrians and vehicles to navigate safely and avoid collisions. However, single-vehicle prediction models frequently fail when faced with blind spots, sensor range limits, and occlusions—such as a pedestrian stepping out from behind a large truck.
While vehicle-to-vehicle (V2V) collaboration allows cars to share information and "see" around obstacles, existing methods typically fuse raw sensor data or high-dimensional feature maps at the perception stage (early or intermediate fusion). This creates massive communication bottlenecks, requires unrealistic network bandwidth, and assumes perfect synchronization between vehicles' local clocks. This paper tackles these limitations by moving collaboration further down the pipeline to the prediction stage.
Methods
The authors propose a late-fusion framework where vehicles share lightweight, sparse trajectory predictions and their associated uncertainties over a standard V2V broadcast channel.
The system architecture treats each vehicle as an independent, asynchronous agent running its own perception-prediction stack. The core components include:
- Agent-Level Prediction Map: Tracks targets as either Category L (locally observed) or Category S (shared-only, meaning they are completely occluded from the ego vehicle but visible to a collaborator).
- Algorithmic Temporal Alignment: Corrects for transmission delays and clock mismatches on the fly without needing joint training or synchronized frame rates.
- Gaussian Process (GP) Regression Fusion: Fuses trajectories coordinate-by-coordinate. For locally seen objects (Category L), a Kalman Filter (KF) gate determines if the vehicle's local tracking uncertainty is high enough to warrant a GP-based residual correction. For unseen objects (Category S), the trajectory is reconstructed entirely from the shared predictions pool.
Datasets
The framework was evaluated across three multi-vehicle datasets:
- OPV2V: A simulated dataset featuring up to 7 collaborating vehicles per scenario.
- DeepAccident: A CARLA-simulated dataset focused on diverse and complex collision scenarios.
- V2V4Real: A real-world dataset collected from two vehicles on urban and highway routes, providing a true test for real-world deployment.
Key Insights and Ideas
- Communication Efficiency via Sparsity: Trajectory coordinates are vastly smaller than neural network feature maps. Fusing at the prediction stage brings the message size well within the 1,500-byte limit of practical 5G NR-V2X protocols, making real-world deployment scalable.
- Asynchronous Independence: By handling time alignment algorithmically rather than through learned delay-compensation networks, the framework allows heterogeneous vehicles with different frame rates and local clocks to collaborate seamlessly.
- The "Recovery" Trade-off: Fusing shared data successfully uncovers previously missed (occluded) objects, which drastically improves overall safety coverage. However, because these newly recovered objects often have higher initial tracking errors, average displacement metrics can occasionally fluctuate slightly even as overall safety improves.
Key Results
- Accuracy & Coverage: Across all datasets, late fusion consistently reduced the miss rate and boosted the Trajectory Success Rate ($\text{TSR}_{0.5}$), defined as the fraction of agents predicted within a 0.5-meter error margin.
- Real-World Validation: On the real-world V2V4Real dataset, collaborative prediction improved the trajectory success rate by 1.69% and 1.22% for the test vehicles compared to individual forecasting.
- Feasibility: The average per-step runtime for the collaborative setup ranged between 43.6 ms and 69.6 ms—well under the 100 ms threshold required for real-time 10 Hz operation.
Limitations and Future Work
- Uncertainty Calibration: The current fusion relies heavily on the accuracy of each vehicle's self-reported prediction uncertainty. Better decoupling of internal model uncertainty from input-driven sensor noise is needed.
- Adaptive Sharing: To scale to dense traffic, future iterations need adaptive sharing policies (e.g., only broadcasting trajectories of vehicles that are closest or most critical to neighbors) to minimize bandwidth.
- GP Computational Overhead: Fitting Gaussian Processes online can become computationally expensive as the number of tracked agents grows. The authors suggest implementing a gating mechanism to filter out noisy tracks before the fusion step.
Summary updated