DSRC: Learning Density-Insensitive and Semantic-Aware Collaborative Representation Against Corruptions

Jingyu Zhang, Yilei Wang, Lang Qian, Peng Sun, Zengwen Li, Sudong Jiang, Maolin Liu, Liang Song

TL;DR

This paper introduces DSRC, a novel method for robust multi-agent collaborative perception in autonomous driving, designed to overcome performance degradation caused by real-world environmental corruptions like fog, snow, and sensor noise. By employing a unique "sparse-to-dense" distillation and semantic-aware learning approach, DSRC significantly improves detection accuracy and robustness compared to existing state-of-the-art methods, even under challenging corrupted conditions.

What Problem Does This Paper Address?

Autonomous vehicles rely heavily on accurate 3D object detection for safe navigation. Multi-agent collaborative perception, where vehicles share sensor data, has shown great promise in enhancing detection capabilities by overcoming individual sensor limitations. However, these systems often perform poorly when faced with real-world environmental challenges such as adverse weather (fog, snow), sensor malfunctions (noise, beam missing), or external disturbances (dust, insects). These "corruptions" can lead to sparse or noisy sensor data, causing significant drops in detection accuracy. Existing collaborative perception methods are typically evaluated on clean datasets and lack robustness in these complex, real-world scenarios. This paper addresses this critical gap by introducing a new benchmark for evaluating corruption robustness and proposing a method, DSRC, specifically designed to learn robust collaborative representations.

Methods

DSRC employs a teacher-student distillation framework to enhance the student model's perception capabilities. The core innovations lie in two key components:

  1. Semantic-Guided Sparse-to-Dense Distillation: This framework tackles the issue of sparse point clouds caused by corruptions. It constructs a "teacher" model that uses a dense, multi-view point cloud, augmented with ground truth bounding boxes to "paint" semantic information onto the points. This dense, semantically rich teacher guides the "student" model, which operates on sparser, real-world data, to learn density-insensitive and semantically aware representations. This distillation happens in three stages: after encoding, after fusion, and after prediction, ensuring knowledge transfer across different levels of processing.
  2. Feature-to-Point Cloud Reconstruction: To further improve the fusion of collaborative information, DSRC incorporates a reconstruction module. This module supervises the student model by tasking it to reconstruct the point cloud from its fused features. The idea is that if the fused features are rich and accurate, they should be able to reconstruct the original scene, thereby enforcing better feature fusion and representation.

During inference, only the student model and the reconstruction module are used, making it computationally efficient.

Datasets

The paper uses two primary datasets for evaluation:

  • OPV2V: A large-scale vehicle-to-vehicle collaborative perception dataset collected using the CARLA simulator and OpenCDA. It includes annotated point clouds and RGB images.
  • DAIR-V2X: A real-world dataset for vehicle-to-infrastructure perception, featuring synchronized LiDAR data from both vehicles and infrastructure.

The authors also introduce a comprehensive benchmark for evaluating corruption robustness, simulating six common corruption types: beam missing, motion blur, fog, snow, crosstalk, and cross sensor.

Key Insights and Ideas

  • Real-world corruptions are a major bottleneck: Standard collaborative perception methods, while impressive on clean data, degrade significantly under realistic environmental conditions. This highlights the need for robustness-focused research.
  • Density and semantics are key to robustness: DSRC's success stems from explicitly addressing both the sparsity of corrupted data (density-insensitive) and the degradation of object information (semantic-aware).
  • Distillation is powerful for transferring robustness: By using a dense, semantically rich teacher model, the student model can learn to generalize better to noisy and sparse inputs.
  • Reconstruction as a fusion supervisor: The feature-to-point cloud reconstruction module provides an effective, task-agnostic way to enforce better fusion of information from multiple agents.
  • More agents generally mean better performance: The study shows that increasing the number of collaborating agents improves performance, though the gains diminish beyond a certain point.

Key Results

  • Outperforms SOTA in clean conditions: DSRC achieves state-of-the-art performance on both OPV2V and DAIR-V2X datasets even in clean environments.
  • Significantly more robust to corruptions: DSRC demonstrates substantial performance improvements across all tested corruption types. For example, it shows a 4.9% improvement in cross-sensor corruption ([email protected]) and 4.02% in crosstalk corruption ([email protected]) on OPV2V compared to other methods.
  • Effective against adverse weather: Under fog conditions, DSRC achieved 8.5% higher [email protected] than the state-of-the-art on OPV2V.
  • Ablation studies confirm design effectiveness: Each of DSRC's core components (Sparse-to-Dense Distillation, Point Cloud Painting, and Point Cloud Reconstruction) contributes to improved performance, with the full DSRC framework yielding the best results.
  • Qualitative results show better detection: Visualizations show DSRC producing more accurate and complete detections, especially in challenging scenarios like scene edges or crowded areas, where other methods miss objects.

Limitations and Future Work

  • Number of agents: While performance improves with more agents, the gains become marginal beyond three agents. Future work could explore more efficient collaboration strategies for larger agent groups.
  • Communication range: The current implementation assumes a fixed communication range (70m). Future work could investigate adaptive or dynamic communication range strategies.
  • Computational cost: Although DSRC is designed for efficiency during inference, the teacher model used during training adds computational overhead. Further research could explore more efficient distillation techniques.
  • New corruption types: While the paper introduces a benchmark for common corruptions, exploring a wider range of novel or complex corruptions could further enhance robustness.

Summary updated

Abstract

As a potential application of Vehicle-to-Everything (V2X) communication, multi-agent collaborative perception has achieved significant success in 3D object detection. While these methods have demonstrated impressive results on standard benchmarks, the robustness of such approaches in the face of complex real-world environments requires additional verification. To bridge this gap, we introduce the first comprehensive benchmark designed to evaluate the robustness of collaborative perception methods in the presence of natural corruptions typical of real-world environments. Furthermore, we propose DSRC, a robustness-enhanced collaborative perception method aiming to learn Density-insensitive and Semantic-aware collaborative Representation against Corruptions. DSRC consists of two key designs: i) a semantic-guided sparse-to-dense distillation framework, which constructs multi-view dense objects painted by ground truth bounding boxes to effectively learn density-insensitive and semantic-aware collaborative representation; ii) a feature-to-point cloud reconstruction approach to better fuse critical collaborative representation across agents. To thoroughly evaluate DSRC, we conduct extensive experiments on real-world and simulated datasets. The results demonstrate that our method outperforms state-of-the-art collaborative perception methods in both clean and corrupted conditions.
BibTeX
@inproceedings{zhang2025dsrc,
  author = {Jingyu Zhang and Yilei Wang and Lang Qian and Peng Sun and Zengwen Li and Sudong Jiang and Maolin Liu and Liang Song},
  title = {{DSRC: Learning Density-Insensitive and Semantic-Aware Collaborative Representation Against Corruptions}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=QAZVVNI1T0},
}
Figure