Multi-Agent Collaborative Perception via Motion-Aware Robust Communication Network

Shixin Hong, Yu Liu, Zhi Li, Shaohui Li, You He

Abstract

Collaborative perception enables autonomous agents to share sensor data to overcome individual sensing limitations such as occlusions and long-range detection gaps. However, real-world deployments face significant challenges due to pose noise from localization errors and motion blur from dynamic objects. This paper presents MRCNet (Motion-aware Robust Communication Network), a framework designed for robust multi-agent collaborative perception. MRCNet incorporates a Multi-scale Robust Fusion (MRF) strategy to mitigate spatial misalignments and a Motion Enhanced Mechanism (MEM) to leverage temporal context for sharpening dynamic object features. Experimental results on V2XSim, OPV2V, and V2XSet demonstrate that MRCNet achieves state-of-the-art performance and superior robustness against noise while maintaining communication efficiency.

1. Introduction

Autonomous driving relies heavily on accurate perception. While single-agent systems are limited by line-of-sight, collaborative perception allows vehicles and infrastructure to share information, effectively "seeing" through obstacles. Despite its potential, current methods often fail in non-ideal conditions. Specifically, pose noise—originating from GPS or IMU inaccuracies—leads to spatial misalignments when fusing data from different agents. Furthermore, motion blur in sensor data complicates the detection of fast-moving objects. MRCNet addresses these issues by focusing on spatial robustness and temporal enhancement.

2. The MRCNet Framework

The architecture of MRCNet is designed to handle the imperfections of real-world communication and localization. It consists of three primary components: feature extraction, robust fusion, and motion enhancement.

The overall architecture of MRCNet, featuring the Multi-scale Robust Fusion (MRF) and Motion Enhanced Mechanism (MEM) modules.

2.1 Multi-scale Robust Fusion (MRF)

To combat pose noise, MRCNet employs the MRF module. Traditional fusion at a single resolution is highly sensitive to coordinate offsets. MRF utilizes Multi-scale Enhanced Aggregation (MEA), which processes features across multiple resolutions. By leveraging high-level semantic features—which are inherently more stable under spatial shifts—the system can provide a "global" anchor to re-align lower-level detailed features. This multi-stage process ensures that even with significant localization errors, the fused representation remains coherent.

2.2 Motion Enhanced Mechanism (MEM)

Dynamic objects often suffer from motion blur or temporary occlusions. The MEM module addresses this by maintaining a historical memory bank of Bird’s-Eye-View (BEV) features. A Gated Motion Unit (GMU) uses deformable convolutions to adaptively align past features with the current frame. This temporal integration allows the network to "sharpen" the perception of moving objects by utilizing information from previous time steps where the object might have been clearer or in a different relative position.

2.3 Communication Efficiency

Bandwidth is a critical constraint in V2X (Vehicle-to-Everything) communication. MRCNet implements a smart feature selection mechanism. Instead of transmitting entire feature maps, it generates a semantic confidence map using channel-wise pooling. This ensures that only the most informative regions for the detection task are shared, reducing communication volume without sacrificing accuracy.

3. Experiments and Evaluation

The performance of MRCNet was validated on three major benchmarks: V2XSim, OPV2V, and V2XSet. V2XSet is particularly important as it includes realistic noise simulations to test robustness.

3.1 Quantitative Results

MRCNet sets a new state-of-the-art (SOTA) on the V2XSim dataset, achieving 85.33% [email protected], outperforming previous models like DiscoNet (83.56%). Similar gains were observed across other datasets, confirming the effectiveness of the proposed modules.

3.2 Robustness to Noise

A key contribution of MRCNet is its resilience to localization errors. In stress tests involving heading errors and positional offsets, MRCNet demonstrated a much slower performance decay compared to existing methods like V2X-ViT and Where2comm.

Performance comparison of MRCNet against various baselines under increasing heading error across three datasets.

As shown in the robustness plots, MRCNet maintains a significant lead in Average Precision (AP) even as the standard deviation of heading error increases to 1.0°. This suggests that the multi-scale fusion strategy successfully compensates for coordinate misalignments that typically cripple collaborative systems.

4. Conclusion

MRCNet provides a comprehensive solution for the practical challenges of collaborative perception. By integrating multi-scale spatial fusion and motion-aware temporal enhancement, it achieves high accuracy and robustness in noisy environments. Future work will explore handling communication latency and model heterogeneity to further bridge the gap between simulation and real-world deployment.

Summary updated

Abstract

Collaborative perception allows for information sharing between multiple agents, such as vehicles and infrastructure, to obtain a comprehensive view of the environment through communication and fusion. Current research on multi-agent collaborative perception systems often assumes ideal communication and perception environments and neglects the effect of real-world noise such as pose noise, motion blur, and perception noise. To address this gap, in this paper, we propose a novel motion-aware robust communication network (MRCNet) that mitigates noise interference and achieves accurate and robust collaborative perception. MRCNet consists of two main components: multi-scale robust fusion (MRF) addresses pose noise by developing cross-semantic multi-scale enhanced aggregation to fuse features of different scales, while motion enhanced mechanism (MEM) captures motion context to compensate for information blurring caused by moving objects. Experimental results on popular collaborative 3D object detection datasets demonstrate that MRCNet outperforms competing methods in noisy scenarios with improved perception performance using less bandwidth. Our code will be released at https://github.com/IndigoChildren/collaborative-perception-MRCNet.
BibTeX
@inproceedings{hong2024multi,
  author = {Shixin Hong and Yu Liu and Zhi Li and Shaohui Li and You He},
  title = {{Multi-Agent Collaborative Perception via Motion-Aware Robust Communication Network}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  doi = {10.1109/cvpr52733.2024.01449},
}
Figure