Abstract
Collaborative perception is a promising approach to enhance the safety of autonomous vehicles (AVs) by enabling them to share sensor data via Vehicle-to-Everything (V2X) communication. While existing methods focus on spatial feature fusion, they often ignore the critical impact of temporal latency and sensor asynchrony. This paper introduces Select2Col, a novel framework that evaluates the Importance of Semantic Information (IoSI) based on both spatial and temporal factors. By intelligently selecting collaborators and utilizing a historical prior hybrid attention mechanism, Select2Col achieves state-of-the-art performance in 3D object detection while maintaining robustness against communication delays.
Introduction: The Challenge of Perceptual Blindness
Autonomous vehicles rely heavily on local sensors like LiDAR and cameras. However, these sensors are limited by line-of-sight occlusions—such as a large truck blocking the view of a pedestrian—and finite detection ranges. Collaborative perception addresses these "blind spots" by allowing vehicles and roadside infrastructure to share extracted features, known as semantic information.
Despite the potential of V2X, real-world communication is rarely perfect. Transmission delays and varying sensor sampling rates mean that the information received by an "ego" vehicle may be outdated (stale). Fusing this stale data without correction can lead to "ghost" detections or misaligned object positions, ultimately compromising safety. Select2Col is designed to address this spatial-temporal mismatch.
Methodology: The Select2Col Framework
The Select2Col architecture is a modular pipeline designed for efficiency and accuracy. It processes raw LiDAR point clouds through four main stages: Encoding, Selection, Fusion, and Decoding.
1. Encoder
The system uses a PointPillars backbone to convert raw 3D point clouds into a 2D representation. This approach is computationally efficient, making it suitable for real-time processing on vehicle hardware.
2. IoSI-based Collaborator Selection
Rather than fusing data from every available agent, Select2Col employs a Selector module. This module uses a lightweight Graph Neural Network (GNN) to calculate the Importance of Semantic Information (IoSI). The GNN considers the spatial coverage of a collaborator's data and its associated communication latency. By downweighting or discarding irrelevant or excessively delayed information, the system reduces noise and optimizes bandwidth usage.
3. HPHA (Historical Prior Hybrid Attention) Fusion
The Fuser module merges the selected features using a two-step attention mechanism:
- Multi-scale Spatial Attention: This module captures correlations between the ego vehicle and its collaborators across different spatial resolutions, ensuring that both fine-grained and global features are preserved.
- Short-term Temporal Attention: This component uses the ego vehicle’s own historical frames as a reference. By comparing incoming collaborative data against its own recent memory, the system can filter out temporal noise and better align stale data with the current environment.
4. Decoder
Finally, the fused information is passed to a decoder that performs classification and bounding box regression to produce the final 3D object detection results.
Evaluation and Results
Select2Col was rigorously tested on three benchmark datasets: OPV2V (simulation), V2XSet (V2I scenarios), and V2V4Real (real-world data).
- Superior Accuracy: On the OPV2V dataset, Select2Col achieved an Average Precision (AP) of 77.65% (at IoU=0.7), outperforming established models like V2VNet (52.96%) and Where2comm.
- Real-World Performance: In the V2V4Real benchmark, which features noisy real-world environments, the model maintained a significant lead with 24.89% AP.
- Resilience to Latency: One of the most critical findings was the model's stability under high latency. Even with a simulated delay of 400ms, Select2Col’s performance remained robust, whereas competing models experienced sharp declines in accuracy.
- Efficiency: With only 8.28 million parameters and an inference time under 100ms, the framework is well-suited for the 10Hz cycle typical of automotive LiDAR systems.
Discussion and Future Work
Select2Col demonstrates that "when" information is recorded is just as important as "where" it comes from. By treating latency as a core variable rather than an external nuisance, the framework achieves a more reliable perception of the environment.
However, challenges remain. The current weighting strategy is sensitive to specific detection thresholds, and the sharing of semantic features raises potential privacy concerns. Future research will focus on refining these selection thresholds and implementing privacy-preserving techniques to prevent the reconstruction of sensitive data from shared feature maps.
Summary updated