Contrastive Learning for Unsupervised Radar Place Recognition

Matthew Gadd, Daniele De Martini, Paul Newman

TL;DR

This paper introduces a novel unsupervised framework for radar-based place recognition, allowing autonomous vehicles to localize themselves without needing massive labeled datasets. By treating temporally successive radar scans as "natural augmentations" in a contrastive learning setup, the authors achieve a new state-of-the-art accuracy of 98.38%, outperforming even some supervised methods.

What Problem Does This Paper Address?

For autonomous vehicles to operate reliably in "all-weather" conditions (fog, rain, snow), traditional cameras and LiDAR often fall short. Scanning Frequency Modulated Continuous-Wave (FMCW) radar is a robust alternative, but it comes with its own headaches: speckle noise, ghosting artifacts, and perceptual aliasing (where different places look identical).

Most existing deep learning solutions for radar localization are supervised, meaning they require expensive, human-annotated ground truth data. The authors tackle the challenge of unsupervised learning: how can a vehicle learn to recognize a location just by looking at the raw radar data it collects while driving, without any external labels?

Methods

The core of the approach is Contrastive Learning. In a typical contrastive setup, the model learns to pull "positive" pairs (different views of the same thing) closer together in a high-dimensional feature space (embedding) while pushing "negative" pairs (different things) further apart.

The authors introduce several clever twists tailored for radar:

  • Motion-Enabled Augmentation: Instead of just using synthetic image transforms (like cropping or flipping), they use scans captured within 2 seconds of each other as positive pairs. This forces the network to learn that two slightly different viewpoints of the same street are, in fact, the same place.
  • Spin Augmenting: To handle the fact that a vehicle might return to a spot from a different direction, they apply random azimuthal shifts (rotations) to the radar data during training, ensuring the learned embeddings are rotationally invariant.
  • Hard Negative Mining: They ensure that "negative" samples in a training batch are at least 2 to 6 seconds away from the current scan to prevent the model from accidentally trying to "push away" scans that are actually from the same immediate vicinity.
  • Distributional Similarity: During inference, they "unfreeze" dropout layers to generate a family of embeddings for a single scan. By measuring the similarity between these distributions (using KL divergence) rather than just single points, they better account for sensor noise and uncertainty.

Datasets

The study utilizes two massive, real-world urban radar datasets:

  1. Oxford Radar RobotCar Dataset: Over 280 km of driving around Oxford, UK. It includes challenging "urban canyons" and vegetation.
  2. MulRan Dataset: Approximately 124 km of driving in South Korea. This dataset is particularly difficult because it includes revisits in opposite directions and subtle lane-level differences.

Key Insights and Ideas

  • Real Noise > Synthetic Noise: The authors found that using temporally successive scans as augmentations provides "real" sensor noise (like speckle and artifacts) that synthetic transforms can't replicate. This makes the model much more robust to actual sensor behavior.
  • Uncertainty Matters: By using stochastic embeddings (via dropout), the model can signal when it is "unsure" about a match. This distributional approach significantly boosts performance in ambiguous environments.
  • Unsupervised can beat Supervised: Surprisingly, this unsupervised method outperformed previous supervised radar localization techniques. This suggests that the sheer volume of data available for unsupervised learning can outweigh the benefits of human labels.

Key Results

  • Top-tier Accuracy: On the Oxford dataset, the system achieved a 98.38% Recall@1 rate, meaning it correctly identified the location on the first try nearly every time.
  • Outperforming Baselines: It significantly outperformed Scan Context (SC), a popular non-learned radar descriptor, especially in "revisiting" scenarios where the vehicle is driving in the opposite direction or in a different lane.
  • Robustness: The model showed a 10%–15% improvement in F1-scores over the most naive contrastive learning approaches, proving that their specific temporal and rotational augmentation strategies are critical.

Limitations and Future Work

While the results are impressive, the authors note that lane-level differences (e.g., driving in the left lane vs. the right lane) still pose a challenge, particularly in the MulRan dataset. The embeddings sometimes struggle to distinguish between these very similar but technically distinct positions.

Future work may involve incorporating GPS/INS data as a "weak" or "pseudo" ground truth to further refine the unsupervised process, or exploring other divergence measures (like Mahalanobis distance) for comparing embedding distributions.

Summary updated

Abstract

We present a method for unsupervised learning of single-scan radar embeddings from sequences of radar images that is suitable for solving the place recognition problem with complex radar data. Our method is based on invariant instance feature learning but is tailored for the task of re-localisation by exploiting for data augmentation the temporal successivity of data as collected by a mobile platform moving through the scene smoothly. We experiment across two prominent urban radar datasets totalling over 400 km of driving and show that we achieve a new radar place recognition state-of-the-art. Specifically, the proposed system proves correct for 98.38 % of the queries that it is presented with over a challenging re-localisation sequence, using only the single nearest neighbour in the learned metric space. We also find that our learned model shows better understanding of out-of-lane loop closures at arbitrary orientation than non-learned radar scan descriptors.
BibTeX
@inproceedings{gadd2021contrastive,
  author = {Matthew Gadd and Daniele De Martini and Paul Newman},
  title = {{Contrastive Learning for Unsupervised Radar Place Recognition}},
  booktitle = {ICAR},
  year = {2021},
  url = {https://openreview.net/forum?id=HD48mJticPu},
}

Loading related content...

Figure