ERMVP: Communication-Efficient and Collaboration-Robust Multi-Vehicle Perception in Challenging Environments

Jingyu Zhang, Kun Yang, Yilei Wang, Hanqi Wang, Peng Sun, Liang Song

Abstract

Collaborative perception among Connected Autonomous Vehicles (CAVs) offers a "god-view" of the environment, overcoming individual sensor limitations like occlusions. However, practical deployment faces significant hurdles: high communication bandwidth requirements, localization errors, and collaborative noise. This paper presents ERMVP, a framework designed for communication efficiency and collaboration robustness. ERMVP utilizes Filter and Merge Feature Sampling (FMS) to reduce data redundancy and Feature Spatial Calibration (FSC) to align feature maps despite noisy GPS data. Experimental results on V2V4Real and OPV2V benchmarks demonstrate that ERMVP achieves superior detection performance while maintaining low communication overhead, outperforming existing state-of-the-art methods.

Introduction

Autonomous vehicles (AVs) are often limited by their immediate line of sight. Collaborative perception allows CAVs to share sensor data, significantly expanding their situational awareness. Despite its potential, three primary challenges persist: bandwidth bottlenecks from sharing high-resolution 3D data, localization errors leading to misaligned feature maps, and the introduction of noise during data fusion. ERMVP addresses these issues through a novel three-stage pipeline focused on efficiency and robustness.

System architecture of ERMVP showing the pipeline from point clouds to detection output.

Methodology

The ERMVP framework consists of three core components:

  • Filter and Merge Feature Sampling (FMS): To minimize communication costs, FMS filters out irrelevant background data and merges similar foreground features. This approach targets intra-class redundancy, ensuring only the most informative features are transmitted.
  • Feature Spatial Calibration (FSC): FSC enhances robustness by treating feature map alignment as a matching problem. Using RANSAC and bipartite matching, it corrects spatial misalignments caused by noisy localization data, effectively "nudging" incoming features into the correct coordinate frame.
  • Attention-based Fusion & Interaction (AFF & AEI): The Attention-based Feature Fusion (AFF) module merges data using local and global attention. Subsequently, the Accuracy Enhanced Feature Interaction (AEI) module uses the ego vehicle's local features as an anchor to refine collaborative data, preventing external noise from degrading local detection accuracy.

Experimental Results

ERMVP was evaluated on the real-world V2V4Real dataset and the simulated OPV2V dataset. To test robustness, Gaussian noise was added to vehicle positions (up to 0.8m) and headings (up to 0.8°).

  • Robustness to Noise: On V2V4Real with high localization noise, ERMVP achieved an Average Precision (AP) of 60.88%, significantly higher than competing methods like V2X-ViT and CoBEVT.
  • Efficiency: ERMVP maintains high performance even at low communication volumes. It consistently stays above the performance-efficiency frontier compared to other state-of-the-art models.
  • Performance Gains: ERMVP improved detection performance by 12% to 18% over single-vehicle perception, validating the benefits of robust collaboration even in challenging environments.
Comparison of Average Precision (AP) vs. Communication Volume on V2V4Real and OPV2V datasets.

Conclusion

ERMVP provides a comprehensive solution for efficient and robust multi-vehicle perception. By intelligently sampling features and calibrating spatial data at the feature level, it overcomes the practical limitations of bandwidth and localization noise. Future work will explore multi-modal integration and scalability for larger vehicle fleets.

Summary updated

Abstract

Collaborative perception enhances perception performance by enabling autonomous vehicles to exchange complementary information. Despite its potential to revolutionize the mobile industry, challenges in various environments, such as communication bandwidth limitations, localization errors and information aggregation inefficiencies, hinder its implementation in practical applications. In this work, we propose ERMVP, a communication-Efficient and collaboration-Robust Multi-Vehicle Perception method in challenging environments. Specifically, ERMVP has three distinct strengths: i) It utilizes the hierarchical feature sampling strategy to abstract a representative set of feature vectors, using less communication overhead for efficient communication; ii) It employs the sparse consensus features to execute precise spatial location calibrations, effectively mitigating the implications of vehicle localization errors; iii) A pioneering feature fusion and interaction paradigm is introduced to integrate holistic spatial semantics among different vehicles and data sources. To thoroughly validate our method, we conduct extensive experiments on real-world and simulated datasets. The results demonstrate that the proposed ERMVP is significantly superior to the state-of-the-art collaborative perception methods.
BibTeX
@inproceedings{zhang2024ermvp,
  author = {Jingyu Zhang and Kun Yang and Yilei Wang and Hanqi Wang and Peng Sun and Liang Song},
  title = {{ERMVP: Communication-Efficient and Collaboration-Robust Multi-Vehicle Perception in Challenging Environments}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  doi = {10.1109/cvpr52733.2024.01195},
}
Figure