Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar

Dan Barnes, Ingmar Posner

Introduction

For autonomous vehicles to operate safely, they require reliable egomotion estimation (odometry) and localization. While cameras and LiDAR are popular, their performance often degrades in adverse weather conditions like rain or fog, or in poor lighting. Radar is immune to these conditions, making it a highly attractive sensor for urban robotics.

However, radar data is notoriously messy, plagued by noise artifacts such as ghost objects, phase noise, speckle, and saturation. Consequently, off-the-shelf keypoint detectors designed for cameras, such as SIFT or ORB, perform poorly. Hand-crafting features for radar is difficult and often suboptimal. This paper introduces a deep learning framework that learns to detect robust keypoints, scores, and descriptors directly from raw radar data by embedding a differentiable pose estimator into the network.

Methods

The authors propose a U-Net style convolutional neural network that takes raw radar scans and predicts three primary outputs: sub-pixel keypoint locations, usefulness scores, and dense descriptors.

The proposed network architecture predicts keypoint locations, scores, and descriptors from raw radar inputs.

The core innovation lies in the training methodology:

  • Differentiable Pose Estimation: Instead of training on proxy tasks, the authors embed a Singular Value Decomposition (SVD)-based point matching and pose estimation algorithm directly inside the neural network.
  • Task-Driven Learning: Because the entire pipeline is differentiable, the network is supervised purely by ground-truth pose errors. It learns to extract keypoints that specifically minimize odometry drift, bypassing human bias.
  • Unified Localisation: Beyond frame-to-frame tracking, the network generates a 248-D global embedding by max-pooling the dense descriptor map. This embedding is used to detect topological loop closures, allowing the system to recognize previously visited locations.

Experimental Results

The model was trained and evaluated on the Oxford Radar RobotCar Dataset, which features complex urban environments. The evaluation spanned approximately 70km of radar data.

Visualisation of robust keypoints detected and matched across radar scans.

Key Insights

  • Learned Noise Filtering: The network's "Score" head naturally learns to assign near-zero scores to moving objects and radar artifacts, while assigning high scores to static infrastructure like walls and buildings. This effectively masks dynamic noise without explicit semantic labels.
  • Feature Generalization: Descriptors learned for local odometry proved robust for global place recognition. Simply max-pooling these descriptors outperformed specialized architectures like NetVLAD.

Performance Metrics

  • Accuracy: The method reduces translational error by 45% (down to 2.058%) and rotational error by 29% (down to 0.0067 deg/m) compared to previous point-based state-of-the-art methods.
  • Speed: The system operates at 28.5 Hz, an order of magnitude faster than previous point-based baselines.
  • Full SLAM: By combining odometry and loop-closure detection into a pose-graph optimization backend (g2o), the authors demonstrated a complete, real-time mapping and localization system.

Conclusion

While this approach does not yet surpass dense (correlation-based) radar odometry in pure accuracy, it offers a significant advantage: it can solve poses at any rotation. This makes it uniquely suitable for metric loop closures where dense methods often fail due to rotation constraints. Future work includes applying this sensor-agnostic architecture to LiDAR or vision and utilizing learned keypoints for downstream tasks like object velocity estimation.

Summary updated

BibTeX
@inproceedings{barnes2020under,
  author = {Dan Barnes and Ingmar Posner},
  title = {{Under the Radar: Learning to Predict Robust Keypoints for Odometry Estimation and Metric Localisation in Radar}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2020},
  doi = {10.1109/icra40945.2020.9196835},
}
Figure