RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehicles

Ramin Nabati, Hairong Qi

Abstract

Region proposal algorithms play an important role in most state-of-the-art two-stage object detection networks by hypothesizing object locations in the image. Nonetheless, region proposal algorithms are known to be the bottleneck in most two-stage object detection networks, increasing the processing time for each image and resulting in slow networks not suitable for real-time applications such as autonomous driving vehicles. In this paper we introduce RRPN, a Radar-based real-time region proposal algorithm for object detection in autonomous driving vehicles. RRPN generates object proposals by mapping Radar detections to the image coordinate system and generating pre-defined anchor boxes for each mapped Radar detection point. These anchor boxes are then transformed and scaled based on the object's distance from the vehicle, to provide more accurate proposals for the detected objects. We evaluate our method on the newly released NuScenes dataset [1] using the Fast R-CNN object detection network [2]. Compared to the Selective Search object proposal algorithm [3], our model operates more than 100x faster while at the same time achieves higher detection precision and recall.

Introduction

In the world of autonomous driving, "two-stage" object detectors (like Fast R-CNN) are highly accurate but notoriously slow. The bottleneck is the Region Proposal Network (RPN)—the first stage that scans the entire image to "guess" where objects might be. Standard vision-based RPNs are computationally heavy, often making them unsuitable for the split-second decision-making required in self-driving cars.

RRPN (Radar Region Proposal Network) addresses this "speed vs. accuracy" trade-off by using Radar’s spatial strengths to tell the camera exactly where to look, bypassing the slow vision-based search entirely. By replacing computationally expensive vision-based proposal methods with Radar-guided anchors, the system achieves a 100x speedup and higher detection accuracy on the NuScenes dataset compared to traditional methods like Selective Search.

Methodology

The authors propose a three-step pipeline to turn raw Radar points into high-quality image proposals:

  1. Perspective Transformation: Radar detections are typically reported in a "bird’s eye view" (distance and azimuth). The RRPN maps these 3D points onto the 2D image coordinate system using a projection matrix derived from camera calibration.
  2. Adaptive Anchor Generation: Simply placing a box on a Radar point isn't enough because Radar points don't always hit the exact center of an object. RRPN generates a set of "anchor boxes" with different aspect ratios and sizes, including translated versions (shifted left, right, or down) to ensure the proposal captures the full extent of the object even if the Radar return is off-center.
  3. Distance Compensation: In an image, objects get smaller as they get further away. RRPN uses the Radar’s precise range data ($d_i$) to scale the anchor boxes dynamically using a learned linear relationship ($S_i = \alpha \frac{1}{d_i} + \beta$).
Adaptive anchor generation strategies including centered and aligned boxes.

Experimental Evaluation

The researchers evaluated RRPN using the NuScenes dataset, which provides synchronized data from 6 cameras and 5 Radars. They created two subsets: NS-F (front-facing data only) and NS-FB (front and back data). The dataset includes 25 object classes, merged into 6 main categories: Car, Truck, Person, Motorcycle, Bicycle, and Bus.

Key Results and Insights

  • Radar as an Attention Mechanism: Instead of the neural network scanning every pixel, the Radar acts as a pointer, allowing the system to ignore irrelevant parts of the image (like clouds or billboards).
  • Massive Speedup: RRPN operates more than 100x faster than the Selective Search algorithm. While Selective Search takes 2–7 seconds per image, RRPN processes 70–90 images per second.
  • Improved Accuracy: When plugged into a Fast R-CNN framework, RRPN achieved higher Mean Average Precision (mAP) and Mean Average Recall (mAR) than the baseline, with significant gains in detecting smaller classes like pedestrians and bicycles.
Object detection results on the NuScenes dataset showing Radar points (red dots) and generated proposals.

Conclusion and Future Work

While RRPN significantly improves speed, it relies heavily on perfect calibration between sensors. Limitations include Radar noise and a lack of vertical localization (height information). Future work may involve refining anchor logic to account for object orientation using Radar's velocity data. Despite these challenges, RRPN demonstrates that physics-based sensor fusion can effectively bypass the computational bottlenecks of traditional vision-only networks.

Summary updated

Abstract

Region proposal algorithms play an important role in most state-of-the-art two-stage object detection networks by hypothesizing object locations in the image. Nonetheless, region proposal algorithms are known to be the bottleneck in most two-stage object detection networks, increasing the processing time for each image and resulting in slow networks not suitable for real-time applications such as autonomous driving vehicles. In this paper we introduce RRPN, a Radar-based real-time region proposal algorithm for object detection in autonomous driving vehicles. RRPN generates object proposals by mapping Radar detections to the image coordinate system and generating pre-defined anchor boxes for each mapped Radar detection point. These anchor boxes are then transformed and scaled based on the object's distance from the vehicle, to provide more accurate proposals for the detected objects. We evaluate our method on the newly released NuScenes dataset [1] using the Fast R-CNN object detection network [2]. Compared to the Selective Search object proposal algorithm [3], our model operates more than 100x faster while at the same time achieves higher detection precision and recall. Code has been made publicly available at https://github.com/mrnabati/RRPN .
BibTeX
@inproceedings{nabati2019rrpn,
  author = {Ramin Nabati and Hairong Qi},
  title = {{RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehicles}},
  booktitle = {ICIP},
  year = {2019},
  url = {https://openreview.net/forum?id=bRWGzS5TJb},
}
Figure