Contrastive Learning for Unsupervised Radar Place Recognition

Matthew Gadd, Daniele De Martini, Paul Newman

Abstract

We learn, in an unsupervised way, an embedding from sequences of radar images that is suitable for solving the place recognition problem with complex radar data. Our method is based on invariant instance feature learning but is tailored for the task of re-localisation by exploiting for data augmentation the temporal successivity of data as collected by a mobile platform moving through the scene smoothly. We experiment across two prominent urban radar datasets totalling over 400 km of driving and show that we achieve a new radar place recognition state-of-the-art. Specifically, the proposed system proves correct for 98.38% of the queries that it is presented with over a challenging re-localisation sequence, using only the single nearest neighbour in the learned metric space. We also find that our learned model shows better understanding of out-of-lane loop closures at arbitrary orientation than non-learned radar scan descriptors.

Introduction

Reliable localization is a cornerstone of autonomous vehicle technology. While cameras and LiDAR are standard, they often fail in adverse weather conditions like heavy rain, fog, or snow. Scanning Frequency Modulated Continuous-Wave (FMCW) radar offers a robust alternative due to its long-range sensing and weather immunity. However, radar data is notoriously difficult to process, characterized by speckle noise, ghosting artifacts, and perceptual aliasing.

Current deep learning approaches for radar place recognition predominantly rely on supervised learning, which requires vast amounts of human-annotated ground truth data. This paper presents an unsupervised framework that leverages raw radar data to learn robust location embeddings, eliminating the need for expensive labeling.

Methodology

The proposed system utilizes a contrastive learning paradigm to map radar scans into a high-dimensional feature space where similar locations are clustered together.

Overview of the contrastive learning framework using temporal radar scans as natural augmentations.

Key Innovations:

  • Motion-Enabled Augmentation: Instead of standard image transforms, the model uses scans captured within a 2-second window as "positive" pairs. This "natural augmentation" forces the network to recognize that slightly different viewpoints of the same scene represent the same physical location.
  • Spin Augmenting: To achieve rotational invariance, random azimuthal shifts are applied to radar data during training. This ensures the model can recognize a location even when approached from a different direction.
  • Hard Negative Mining: To prevent the model from collapsing, "negative" samples are selected from scans at least 2 to 6 seconds away from the anchor, ensuring distinct locations are pushed apart in the embedding space.
  • Distributional Similarity: During inference, the system uses dropout to generate a distribution of embeddings for each scan. By comparing these distributions via KL divergence, the model accounts for sensor noise and uncertainty more effectively than point-based comparisons.

Experimental Results

The model was evaluated on two large-scale datasets: the Oxford Radar RobotCar Dataset (280 km) and the MulRan Dataset (124 km). These datasets provide diverse urban environments and challenging revisit scenarios, including opposite-direction travel.

Performance comparison across different database candidates (N), showing high recall rates for the proposed method.

The system achieved a state-of-the-art 98.38% Recall@1 on the Oxford dataset. It significantly outperformed traditional non-learned descriptors like Scan Context (SC), particularly in scenarios involving lane-level changes or varying vehicle orientations. The use of temporal successivity as a natural augmentation proved more effective than synthetic noise, as it captures the actual physical characteristics of the sensor.

Conclusion and Future Work

This research demonstrates that unsupervised contrastive learning can match or exceed the performance of supervised methods for radar place recognition. By treating the temporal flow of data as a source of supervision, the model learns a robust representation of the environment.

While the system excels in most scenarios, distinguishing between very similar parallel lanes remains a challenge. Future research will explore integrating weak supervision from GPS/INS data and investigating alternative distance metrics for distributional embeddings to further enhance localization precision.

Summary updated

Abstract

We learn, in an unsupervised way, an embedding from sequences of radar images that is suitable for solving the place recognition problem with complex radar data. Our method is based on invariant instance feature learning but is tailored for the task of re-localisation by exploiting for data augmentation the temporal successivity of data as collected by a mobile platform moving through the scene smoothly. We experiment across two prominent urban radar datasets totalling over 400 km of driving and show that we achieve a new radar place recognition state-of-the-art. Specifically, the proposed system proves correct for 98.38% of the queries that it is presented with over a challenging re-localisation sequence, using only the single nearest neighbour in the learned metric space. We also find that our learned model shows better understanding of out-of-lane loop closures at arbitrary orientation than non-learned radar scan descriptors.
BibTeX
@inproceedings{gadd2021contrastive,
  author = {Matthew Gadd and Daniele De Martini and Paul Newman},
  title = {{Contrastive Learning for Unsupervised Radar Place Recognition}},
  booktitle = {ICAR},
  year = {2021},
  url = {https://openreview.net/forum?id=HD48mJticPu},
}
Figure