Improved Visual Localization via Graph Smoothing

Carlos Eduardo Rosar Kós Lassance, Yasir Latif, Ravi Garg, Vincent Gripon, Ian D. ReidarXiv 2019 · arXiv

Abstract

Visual-Based Localization (VBL) is a critical task for autonomous systems, yet it remains highly sensitive to environmental appearance noise. This paper introduces a post-processing framework that leverages Graph Signal Processing (GSP) to refine image-retrieval-based localization. By constructing a graph based on available metadata—such as GPS coordinates and temporal sequences—the authors apply graph smoothing to latent feature vectors. This approach acts as a denoising filter, significantly reducing localization errors without requiring retraining of the underlying neural networks. Empirical results on large-scale datasets show a reduction in median error from 110m to 22m on challenging test scenarios.

Introduction

Determining a camera's precise pose from a single image is typically approached via direct regression or retrieval-based methods. While retrieval methods generalize better to unseen environments, they often fail due to "appearance noise"—transient changes in lighting, weather, or seasonal variations.

Most existing methods treat images as isolated data points. However, in practical applications like robotics, images are captured in sequences and often accompanied by coarse metadata. This paper explores a novel way to utilize this extra information to "clean up" noisy image representations using graph-based diffusion.

Methods

The core of the proposed framework is the application of Graph Smoothing to the latent feature vectors (e.g., NetVLAD representations) of images.

Graph Construction

A graph is constructed where each vertex represents an image. Edges are defined by three distinct similarity sources:

  1. Metric Distance ($W_{dist}$): Derived from available GPS coordinates.
  2. Sequence ($W_{seq}$): Based on the temporal order of acquisition, assuming images taken close in time are physically nearby.
  3. Latent Similarity ($W_{latent\_sim}$): Standard cosine similarity between deep features.

The Smoothing Operation

Image features are treated as a "signal" on the graph. By applying a normalized adjacency matrix ($A^m$) to the feature matrix, the system performs a diffusion-like operation. This effectively averages an image's features with those of its spatial and temporal neighbors, filtering out outliers caused by transient environmental factors.

Conceptual illustration of graph smoothing $h_G(s)$ applied to image features on a graph.

Results and Discussion

The authors evaluated the method on large-scale datasets from Adelaide and Sydney, representing complex real-world conditions. The pre-trained NetVLAD model was used without further fine-tuning on these specific locations to test generalization.

Quantitative results comparing the baseline (None) with various graph smoothing configurations (GS Support, GS Query, GS S+Q).

As shown in the table above, the improvements were most dramatic in the "Test" scenarios (Adelaide dataset). The baseline median distance error of 110.66m was reduced to 22.81m when applying smoothing to both the support database and the query (GS S+Q).

Key Insights

  • Denoising Effect: Appearance variations act as high-frequency noise in the latent space; graph smoothing serves as a low-pass filter to enforce spatial consistency.
  • Optimal Smoothing: The authors found that applying the operation twice ($m=2$) is the "sweet spot." Excessive smoothing eventually causes distinct locations to become indistinguishable.
  • Plug-and-Play: The method requires no retraining and can be integrated into any existing retrieval-based localization pipeline as a lightweight inference-time adjustment.

Conclusion

Graph smoothing provides a robust mechanism for improving visual localization by exploiting the inherent structure of image sequences and spatial metadata. While the method depends on the availability of such metadata, it offers a significant performance boost for autonomous vehicles and robotics platforms. Future work will investigate end-to-end training of the graph parameters and enforcing temporal consistency during the final pose estimation step.

Summary updated

Abstract

Vision based localization is the problem of inferring the pose of the camera given a single image. One solution to this problem is to learn a deep neural network to infer the pose of a query image after learning on a dataset of images with known poses. Another more commonly used approach rely on image retrieval where the query image is compared against the database of images and its pose is inferred with the help of the retrieved images. The latter approach assumes that images taken from the same places consists of the same landmarks and, thus would have similar feature representations. These representation can be learned using full supervision to be robust to different variations in capture conditions like time of the day and weather. In this work, we introduce a framework to enhance the performance of these retrieval based localization methods by taking into account the additional information including GPS coordinates and temporal neighbourhood of the images provided by the acquisition process in addition to the descriptor similarity of pairs of images in the reference or query database which is used traditionally for localization. Our method constructs a graph based on this additional information and use it for robust retrieval by smoothing the feature representation of reference and/or query images. We show that the proposed method is able to significantly improve the localization accuracy on two large scale datasets over the baselines.
BibTeX
@misc{lassance2019improved,
  author = {Carlos Eduardo Rosar Kós Lassance and Yasir Latif and Ravi Garg and Vincent Gripon and Ian D. Reid},
  title = {{Improved Visual Localization via Graph Smoothing}},
  year = {2019},
  eprint = {1911.02961},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.02961},
}
Figure