RICCARDO: Radar Hit Prediction and Convolution for Camera-Radar 3D Object Detection

Yunfei Long, Abhinav Kumar, Xiaoming Liu, Daniel D. Morris

Abstract

RICCARDO (Radar Hit Prediction and Convolution) is a novel framework designed to enhance 3D object detection by explicitly modeling the expected distribution of radar returns. By treating these predicted distributions as convolution kernels and matching them against actual radar measurements, the system significantly refines object depth estimation. RICCARDO achieves state-of-the-art performance on the nuScenes benchmark, demonstrating that physical intuition about radar reflection patterns can effectively bridge the gap between sparse radar data and rich camera imagery.

Introduction

In autonomous driving, precise 3D object detection is critical. While LiDAR provides high-fidelity spatial data, its cost remains a barrier to widespread adoption. Camera-radar fusion offers a more economical alternative, yet it faces significant challenges: cameras struggle with depth-scale ambiguity, and radar produces sparse, noisy point clouds.

Most existing fusion methods rely on implicit learning within "black-box" neural networks. However, radar reflections are not stochastic; they typically bounce off specific structural elements of a vehicle, such as bumpers or wheels, depending on the sensor's viewpoint. RICCARDO addresses the limitations of implicit fusion by explicitly modeling these "hit" patterns, transforming radar's perceived "messiness" into a predictable signal for depth refinement.

Methods

The RICCARDO framework operates through a structured three-stage pipeline:

  1. Radar Hit Prediction (RIC) Model: A neural network predicts a 2D Bird’s-Eye View (BEV) probability map of likely radar reflections. This prediction is conditioned on the object’s category, size, orientation, and distance, as initially estimated by a monocular detector.
  2. Matching via Convolution: The predicted RIC map serves as a convolution kernel. Since monocular cameras are laterally accurate but depth-deficient, the system slides this kernel along the radial (depth) axis, calculating similarity scores between the predicted pattern and actual binned radar measurements.
  3. Candidate Selector: A final Multi-Layer Perceptron (MLP) fusion stage rescores position candidates by weighing the initial camera detection confidence against the radar matching scores.
The three-stage RICCARDO pipeline architecture, illustrating the flow from monocular detection and radar accumulation to final candidate selection.

Key Insights

A core innovation of RICCARDO is the use of object-centric coordinates for the RIC model. By predicting distributions relative to the object itself, the model efficiently learns how hit patterns evolve with changing viewpoints. Unlike simple geometric models, RICCARDO captures the complex nature of radar, which may penetrate objects or reflect off the undercarriage, concentrating hits at the front of a vehicle rather than forming a clean outline.

Results and Discussion

The researchers evaluated RICCARDO on the nuScenes dataset, a comprehensive benchmark for autonomous driving. RICCARDO achieved state-of-the-art results for camera-radar fusion, reaching a nuScenes Detection Score (NDS) of 0.695 and a mean Average Precision (mAP) of 0.630.

The most impactful improvement was observed in depth accuracy. The mean Average Translation Error (mATE) was reduced from 0.425 (monocular baseline) to 0.363. This demonstrates the framework's efficacy in correcting the inherent depth errors of camera-only systems. The method proved robust across all 10 object categories and showed significant gains in low-light conditions where cameras typically underperform.

Visual breakdown of the RICCARDO process: (a) initial 3D bounding box, (b) predicted radar hit distribution, (c) radar points in BEV, and (d) the resulting probability density function for range refinement.

Conclusion

RICCARDO demonstrates that explicit physical modeling of sensor characteristics can outperform purely data-driven fusion approaches. While currently limited by its dependency on initial monocular detections and a 1D radial search, future iterations could expand to 2D spatial searches and incorporate environmental variables to further enhance robustness in diverse driving conditions.

Summary updated

BibTeX
@inproceedings{long2025riccardo,
  author = {Yunfei Long and Abhinav Kumar and Xiaoming Liu and Daniel D. Morris},
  title = {{RICCARDO: Radar Hit Prediction and Convolution for Camera-Radar 3D Object Detection}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02075},
}
Figure