Introduction
Collaborative perception enables autonomous vehicles to overcome line-of-sight limitations by sharing sensor data with other vehicles or roadside infrastructure. However, real-world deployment faces a significant "domain gap" challenge: different manufacturers use diverse AI models and proprietary training data. Current state-of-the-art methods often assume a "white box" scenario where all agents share the same model architecture and weights. This paper proposes a novel framework for intermediate collaboration—sharing deep features—between unknown agents while preserving privacy and handling localization noise.
Methodology
The proposed framework shifts the collaborative perception paradigm from a "white box" to a "black box" approach, which is far more realistic for a fragmented automotive market. It consists of two primary components: Multi-scale Feature Fusion with Deformable Attention (MSD) and the Plug-in Domain Adapter (PLDA).
Multi-scale Feature Fusion with Deformable Attention (MSD)
To handle discretization errors caused by GPS noise or communication delays, the authors developed MSD. This module uses a pyramidal structure to fuse features across multiple resolutions. A core innovation within this module is the Deformable Attention Block (DAB).
Unlike standard attention mechanisms that look at fixed points, DAB learns to "sample" the most relevant features from a collaborator’s map. This effectively re-aligns data that may be slightly off due to sensor lag or positioning errors, making the system robust to real-world noise.
Plug-in Domain Adapter (PLDA)
To bridge the feature-level gap between different models without sharing private weights, the PLDA acts as a lightweight "translator." It transforms features from a collaborator's domain into the ego vehicle's domain. Crucially, PLDA is trained on a shared public dataset, ensuring that proprietary model weights and private training data remain confidential and independent.
Experimental Results
The researchers evaluated their framework on two major real-world benchmarks: DAIR-V2X (vehicle-infrastructure cooperation) and V2V4Real (vehicle-to-vehicle perception).
Performance and Efficiency
- Superior Accuracy: MSD outperformed existing state-of-the-art (SOTA) models by 2.8% on DAIR-V2X and 6.7% on V2V4Real (measured by Average Precision at 0.7 IoU).
- Heterogeneous Collaboration: With the PLDA adapter, the system's performance in scenarios with different models was able to match or even surpass the performance of "joint training."
- Speed: The PLDA adapter is highly efficient, adding only about 10.18ms of inference time, which is roughly 3x faster than competing domain adaptation methods.
Visualizing the Domain Gap
The necessity of this approach is highlighted by the visual differences in feature maps generated by different models. Even identical architectures can produce inconsistent internal representations if trained on different datasets.
Robustness to Noise
In simulations with GPS noise up to 0.5m, the multi-scale fusion approach maintained significantly higher accuracy than other methods. By fusing features at higher scales (lower resolution), the system provides a larger spatial "buffer," making it naturally more resilient to positioning errors.
Conclusion and Future Work
This research successfully bridges the gap between different LiDAR-based models while maintaining privacy. While highly effective, the authors acknowledge that future work should explore cross-modality collaboration (e.g., LiDAR to camera) and improve performance in extremely high-latency or highly dynamic environments.
Summary updated