AsymLoc: Towards Asymmetric Feature Matching for Efficient Visual Localization

Mohammad Omama, Gabriele Berton, Eric Foxlin, Yelin Kim

TL;DR

AsymLoc introduces an asymmetric framework for visual localization on resource-constrained edge devices by splitting the computational workload: a large, powerful "Teacher" model processes database maps offline, while an ultra-lightweight "Student" model runs online on-device. By distilling joint detector-descriptor representations, AsymLoc enables parameter-less, real-time feature matching that retains up to 95% of the teacher's accuracy while using models up to 25× smaller.

What Problem Does This Paper Address?

Visual localization—estimating a camera's precise 6-DoF position relative to a pre-mapped environment—is foundational for robotics and AR/VR wearables like smart glasses. However, running state-of-the-art localization models directly on edge devices is heavily bottlenecked by strict battery life and thermal dissipation constraints.

While researchers often deploy smaller, compressed models to save compute, doing so typically incurs a severe drop in localization accuracy. Alternatively, bridging the performance gap with learned matchers (like LightGlue) introduces massive parameter and latency overheads, defeating the purpose of a compact model.

AsymLoc tackles a clever workaround: asymmetric localization. Since database map images are static, they can be processed ahead of time offline using a large, highly accurate model. The query images, however, must be processed online using a tiny model. The core challenge is making features extracted from two completely different networks compatible enough to be matched quickly and accurately without heavy, learned matchers.

Figure 1. AsymLoc bridges the gap between powerful database models and lightweight on-device localization. By explicitly modeling teacher–student asymmetry, AsymLoc enables compact query models to perform real-time localization on edge platforms such as smart glasses, drones, and single-board computers, while larger teacher models process the pre-mapped database im- ages offline. This design delivers near-teacher accuracy with up to 25× smaller models and a fraction of the compute cost.

Methods

AsymLoc introduces a novel distillation framework that forces a compact Student model to produce features natively compatible with a frozen Teacher model. Rather than supervising keypoint detection and descriptor extraction independently, AsymLoc aligns them in a joint detector-descriptor space through two primary objectives:

  1. Geometric Matching Loss: Using image pairs with known geometric relationships (homographies), the framework constructs a bi-directional mutual matching matrix. The student is trained to maximize the probability of true geometric correspondences, filtered by the teacher's high-confidence keypoints.
  2. Joint Detector-Descriptor Distillation Loss: The authors construct detector-weighted similarity matrices for both student-teacher and teacher-teacher image pairs. By minimizing the Kullback-Leibler (KL) divergence between these distributions, the student learns exactly how the teacher's detection confidence modulates its descriptor similarities.

At inference time, the student model runs on the query image, and the features are matched against the pre-computed teacher database features using simple, parameter-less Mutual Nearest Neighbor (MNN) matching—completely eliminating the need for heavy learned matching networks.

Figure 2. AsymLoc Training Pipeline. Given a pair of images (A, B) with known homography, the teacher model T processes image A, while image B is processed by both the teacher T and the student S. Each network produces N keypoints with corresponding detector confidence and descriptors. The teacher outputs from A and the student outputs from B are combined to form the Mutual Matching Matrix (Sec. 3.2), which is used to compute the geometric matching loss. In parallel, we construct two detector-weighted similarity matrices: one with the teacher outputs of A and the student outputs of B, and the other with the teacher outputs of A and the teacher outputs of B. These matrices form two joint detector–descriptor similarity spaces (Sec. 3.3); their distributions are then aligned through a distillation loss.

Datasets

The models were trained using synthetic image pairs generated from the COCO dataset. Evaluation spanned five diverse benchmarks to test indoor, outdoor, and multi-domain generalization:

  • HPatches: Evaluates homography estimation under severe illumination and viewpoint changes.
  • ScanNet: An indoor dataset used for evaluating relative pose estimation accuracy.
  • IMC2022: A challenging outdoor landmark dataset.
  • Aachen Day-Night: A full-scale outdoor localization benchmark featuring extreme day-to-night appearance variations.
  • MegaDepth: Used for additional cross-dataset validation (detailed in the appendix).

Key Insights and Ideas

  • Asymmetry is a Free Lunch for Inference: Exploiting the offline nature of map building allows engineers to offload half of the localization pipeline's complexity to the cloud or a server, leaving only the query processing on the edge device.
  • Naïve Distillation Fails for Local Features: Simply forcing a student's descriptors to match a teacher's descriptors via cosine distance yields poor results. Local feature matching is highly dependent on where the network looks.
  • The "Aha" Moment — Joint Modulation: The breakthrough comes from weighting the descriptor similarity matrices by the keypoint detection scores during distillation. This teaches the student model to prioritize learning the representations of highly matchable regions, mimicking the teacher's selective attention.

Key Results

  • Near-Teacher Performance: On the Aachen Day-Night dataset, AsymLoc recovers up to 95.5% of the accuracy of a full SiLK teacher model and 93% of a SuperPoint teacher model.
  • Massive Efficiency Gains: The framework successfully aligns ultra-compact student models as small as 0.04M parameters (25× smaller than the teacher) while operating at a fraction of the GFLOPs and maintaining a flat Pareto efficiency curve.
  • Baseline Superiority: AsymLoc consistently outperformed standard symmetric small models and adapted state-of-the-art asymmetric image retrieval baselines (like CSD and D3Still) across all tested datasets.

Limitations and Future Work

  • Teacher Dependency: The student's performance upper bound is strictly capped by the quality of the chosen teacher model. If the teacher struggles in a specific environment, the student will too.
  • Geometric Training Requirements: The distillation process relies heavily on ground-truth geometric correspondences (homographies or epipolar lines), meaning high-quality 3D or warped training data is mandatory.
  • Future Directions: The authors suggest extending this asymmetric paradigm beyond sparse local features to dense matching methods and end-to-end transformer architectures.

Summary updated

Abstract

Precise and real-time visual localization is critical for applications like AR/VR and robotics, especially on resource-constrained edge devices such as smart glasses, where battery life and heat dissipation can be a primary concerns. While many efficient models exist, further reducing compute without sacrificing accuracy is essential for practical deployment. To address this, we propose asymmetric visual localization: a large Teacher model processes pre-mapped database images offline, while a lightweight Student model processes the query image online. This creates a challenge in matching features from two different models without resorting to heavy, learned matchers. We introduce AsymLoc, a novel distillation framework that aligns a Student to its Teacher through a combination of a geometry-driven matching objective and a joint detector-descriptor distillation objective, enabling fast, parameter-less nearest-neighbor matching. Extensive experiments on HPatches, ScanNet, IMC2022, and Aachen show that AsymLoc achieves up to 95% of the teacher's localization accuracy using an order of magnitude smaller models, significantly outperforming existing baselines and establishing a new state-of-the-art efficiency-accuracy trade-off.
BibTeX
@inproceedings{omama2025asymloc,
  author = {Mohammad Omama and Gabriele Berton and Eric Foxlin and Yelin Kim},
  title = {{AsymLoc: Towards Asymmetric Feature Matching for Efficient Visual Localization}},
  booktitle = {CroCoDL @ ICCV},
  year = {2025},
  url = {https://openreview.net/forum?id=TZ30TeYsBu},
}
Figure