Collaborative Trajectory Prediction via Late Fusion

Nadya Abdel Madjid, Murad Mebrahtu, Zakhar Yagudin, Bilal Hassan, Naoufel Werghi, Jorge Dias, Dzmitry Tsetserukou, Majid KhonjiarXiv 2026 · arXiv

TL;DR

This paper introduces a model-agnostic, late-fusion framework for collaborative vehicle-to-vehicle (V2V) trajectory prediction. By sharing sparse, asynchronous trajectory forecasts instead of heavy perception feature maps, the proposed method drastically reduces communication overhead while significantly improving prediction accuracy and safety coverage under severe occlusions.

What Problem Does This Paper Address?

Autonomous vehicles (AVs) rely heavily on predicting the future paths of nearby pedestrians and vehicles to navigate safely and avoid collisions. However, single-vehicle prediction models frequently fail when faced with blind spots, sensor range limits, and occlusions—such as a pedestrian stepping out from behind a large truck.

While vehicle-to-vehicle (V2V) collaboration allows cars to share information and "see" around obstacles, existing methods typically fuse raw sensor data or high-dimensional feature maps at the perception stage (early or intermediate fusion). This creates massive communication bottlenecks, requires unrealistic network bandwidth, and assumes perfect synchronization between vehicles' local clocks. This paper tackles these limitations by moving collaboration further down the pipeline to the prediction stage.

Figure 1: Occlusion scenarios, where collaboration between vehicles is useful. Left scenario: a fully occluded agent is not observable to the ego vehicle (e.g., a pedestrian hidden behind a truck), motivating the need for external information to avoid collision. Right scenario: agent a2 is partially occluded for iv1 by agent a1, degrading tracking and prediction, while iv2 has a clear view and can estimate a2’s future trajectory more reliably.

Methods

The authors propose a late-fusion framework where vehicles share lightweight, sparse trajectory predictions and their associated uncertainties over a standard V2V broadcast channel.

The system architecture treats each vehicle as an independent, asynchronous agent running its own perception-prediction stack. The core components include:

  1. Agent-Level Prediction Map: Tracks targets as either Category L (locally observed) or Category S (shared-only, meaning they are completely occluded from the ego vehicle but visible to a collaborator).
  2. Algorithmic Temporal Alignment: Corrects for transmission delays and clock mismatches on the fly without needing joint training or synchronized frame rates.
  3. Gaussian Process (GP) Regression Fusion: Fuses trajectories coordinate-by-coordinate. For locally seen objects (Category L), a Kalman Filter (KF) gate determines if the vehicle's local tracking uncertainty is high enough to warrant a GP-based residual correction. For unseen objects (Category S), the trajectory is reconstructed entirely from the shared predictions pool.
Figure 2: System architecture of the proposed collaborative trajectory prediction framework. (a) Broadcast–listen communication setup and packet structure used to share predicted trajectories between vehicles. (b) Agent-level prediction map update: the perception–prediction stack adds, updates, and removes nodes; after data association, the prediction pools of Category L (locally observed) nodes are updated, while Category S (shared-only) nodes are added or updated. The fusion module combines trajectories for Category L and Category S nodes and resets their prediction pools after each fusion step.

Datasets

The framework was evaluated across three multi-vehicle datasets:

  • OPV2V: A simulated dataset featuring up to 7 collaborating vehicles per scenario.
  • DeepAccident: A CARLA-simulated dataset focused on diverse and complex collision scenarios.
  • V2V4Real: A real-world dataset collected from two vehicles on urban and highway routes, providing a true test for real-world deployment.

Key Insights and Ideas

  • Communication Efficiency via Sparsity: Trajectory coordinates are vastly smaller than neural network feature maps. Fusing at the prediction stage brings the message size well within the 1,500-byte limit of practical 5G NR-V2X protocols, making real-world deployment scalable.
  • Asynchronous Independence: By handling time alignment algorithmically rather than through learned delay-compensation networks, the framework allows heterogeneous vehicles with different frame rates and local clocks to collaborate seamlessly.
  • The "Recovery" Trade-off: Fusing shared data successfully uncovers previously missed (occluded) objects, which drastically improves overall safety coverage. However, because these newly recovered objects often have higher initial tracking errors, average displacement metrics can occasionally fluctuate slightly even as overall safety improves.

Key Results

  • Accuracy & Coverage: Across all datasets, late fusion consistently reduced the miss rate and boosted the Trajectory Success Rate ($\text{TSR}_{0.5}$), defined as the fraction of agents predicted within a 0.5-meter error margin.
  • Real-World Validation: On the real-world V2V4Real dataset, collaborative prediction improved the trajectory success rate by 1.69% and 1.22% for the test vehicles compared to individual forecasting.
  • Feasibility: The average per-step runtime for the collaborative setup ranged between 43.6 ms and 69.6 ms—well under the 100 ms threshold required for real-time 10 Hz operation.

Limitations and Future Work

  • Uncertainty Calibration: The current fusion relies heavily on the accuracy of each vehicle's self-reported prediction uncertainty. Better decoupling of internal model uncertainty from input-driven sensor noise is needed.
  • Adaptive Sharing: To scale to dense traffic, future iterations need adaptive sharing policies (e.g., only broadcasting trajectories of vehicles that are closest or most critical to neighbors) to minimize bandwidth.
  • GP Computational Overhead: Fitting Gaussian Processes online can become computationally expensive as the number of tracked agents grows. The authors suggest implementing a gating mechanism to filter out noisy tracks before the fusion step.

Summary updated

Abstract

Predicting future trajectories of surrounding traffic agents is critical for safe autonomous navigation and collision avoidance. Despite all advances in the trajectory forecasting realm, the prediction models remains vulnerable to uncertainty caused by occlusions, limited sensing range, and perception errors. Collaborative vehicle-to-vehicle (V2V) approaches help reduce this uncertainty by sharing complementary information. Existing collaborative trajectory prediction methods typically fuse feature maps at the perception stage to construct a holistic scene view. Further this holistic representation is decoded into the future trajectories. Such design incurs substantial communication overhead due to the exchange of high-dimensional feature representations and often assumes idealized bandwidth and synchronization, limiting practical deployment. We address these limitations by shifting collaboration from perception to the prediction module and introducing a late-fusion framework for shared forecasts. The framework is model-agnostic and treats collaborating vehicles as independent asynchronous agents. We evaluate the approach on the OPV2V, V2V4Real, and DeepAccident datasets, comparing individual and collaborative forecasting. Across all datasets, late fusion consistently reduces miss rate and improves trajectory success rate ($\mathrm{TSR}_{0.5}$), defined as the fraction of ground-truth agents with final displacement error below 0.5 m. On the real-world V2V4Real dataset, collaborative prediction improves the success rate by $1.69\%$ and $1.22\%$ for both intelligent vehicles, respectively, compared with individual forecasting.
BibTeX
@misc{madjid2026collaborative,
  author = {Nadya Abdel Madjid and Murad Mebrahtu and Zakhar Yagudin and Bilal Hassan and Naoufel Werghi and Jorge Dias and Dzmitry Tsetserukou and Majid Khonji},
  title = {{Collaborative Trajectory Prediction via Late Fusion}},
  year = {2026},
  eprint = {2604.22973},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.22973},
}

Loading related content...

Figure