Introduction
Autonomous Driving Systems (ADS) rely on a multi-modal sensor suite, typically including cameras, LiDAR, and radar. While cameras offer rich semantic information, they are susceptible to adverse weather and lack inherent depth perception. Radar, conversely, is robust to environmental conditions but is often processed into sparse point clouds, discarding valuable contextual data.
A significant challenge in sensor fusion is the geometric discrepancy between camera perspective views and radar's polar (Range-Azimuth) coordinate system. Many current solutions employ computationally intensive transformers to bridge this gap, which can be prohibitive for real-time edge deployment. This paper introduces REFNet, a resource-efficient fusion network that pre-aligns camera data into a polar Bird’s-Eye View (BEV) to match the radar's natural domain, achieving high accuracy at nearly 60 FPS.
Methodology
The core innovation of REFNet lies in its "pre-alignment" strategy, which simplifies the fusion task for the neural network.
Sensor Alignment and Processing
Instead of requiring the network to learn complex coordinate transformations, the authors implement a geometric preprocessing pipeline:
- Camera Stream: Perspective images are projected into a Cartesian BEV and subsequently transformed into a BEV Polar domain. This ensures that camera features are spatially aligned with the radar's Range-Azimuth format before feature extraction.
- Radar Stream: The network processes raw Range-Doppler (RD) tensors. A channel-swapping strategy is utilized to recover angular information, converting RD data into Range-Azimuth (RA) features.
Architecture
REFNet utilizes a dual-stream architecture based on ResNet-50 encoders. One branch processes the polar-transformed camera images, while the other handles the radar data. The resulting features from both decoders are fused via channel concatenation. A specialized detection head then performs classification and regression to localize vehicles in the range-azimuth space.
Experimental Setup
The model was evaluated using the RADIal dataset, which provides 2 hours of synchronized data from automotive-grade sensors. Crucially, RADIal includes raw Analog-to-Digital Converter (ADC) signals and Range-Doppler views, enabling the study of low-level radar fusion. The dataset contains approximately 25,000 frames, with over 8,000 labeled frames for vehicle detection.
Key Results
REFNet demonstrates a superior balance of localization precision and computational efficiency.
- Accuracy: On the RADIal dataset, REFNet achieved state-of-the-art localization with a Range Error of 0.11m and an Angle Error of 0.09°. It reached a competitive 93.49% F1-score.
- Efficiency: The model operates at 58.91 FPS on an NVIDIA RTX A6000. It maintains a significantly smaller memory footprint than comparable models like EchoFusion, using nearly four times fewer parameters.
- Ablation Study: The authors confirmed that using Polar BEV input for the camera stream (93.49% F1) significantly outperforms Cartesian BEV input (88.41% F1), validating the importance of domain alignment.
Conclusion
By adopting the polar coordinate system as a "common language" for sensor fusion, REFNet eliminates the need for heavy transformer-based cross-attention mechanisms. This approach preserves the rich information in raw radar data while maintaining the real-time performance necessary for autonomous driving. Future work will focus on expanding detection to more diverse object classes and improving robustness to camera calibration errors.
Summary updated