Abstract
Visual localization typically relies on a two-stage hierarchical pipeline: global image retrieval followed by local feature matching. However, maintaining separate models for these tasks is computationally expensive and redundant. We present SuperGF, a Transformer-based framework that generates global image descriptors by aggregating local features already extracted for matching. By bridging the feature-level gap between retrieval and matching, SuperGF achieves state-of-the-art localization accuracy while significantly reducing memory and computational overhead.
Introduction
Visual localization—the task of determining a camera's precise 6-Degree-of-Freedom (6-DoF) pose—is a cornerstone of robotics and autonomous systems. The standard approach involves a "hierarchical" process: first, image retrieval identifies candidate images from a database using global features; second, precise geometric correspondences are established using local features.
A significant bottleneck in this pipeline is the "feature-level gap." Retrieval requires high-level semantic understanding (e.g., identifying a street), whereas matching demands spatial invariance and pixel-level precision (e.g., identifying a specific corner). Consequently, most systems deploy two heavy, independent neural networks, such as NetVLAD for global and SuperPoint for local features. SuperGF addresses this redundancy by "upcycling" local features into a global descriptor using a Transformer architecture, treating global feature extraction as an aggregation problem rather than a separate vision task.
Methodology
SuperGF introduces a holistic model that leverages the global receptive field of Transformers to synthesize global context from local descriptors. The framework is proposed in two distinct architectural variants to suit different computational needs.
Dense and Sparse Architectures
- Dense Version: This variant operates on intermediate feature maps from a local feature encoder. A CNN-based tokenizer downsamples these maps before they are processed by Transformer encoder layers to capture long-range dependencies.
- Sparse Version: Designed for extreme efficiency, this version operates directly on sparse keypoints (descriptors, positions, and confidence scores). It employs Slot Attention Clustering to group a variable number of points into a fixed set of "cluster tokens," which are then refined by the Transformer.
Both versions conclude with Generalized Mean (GeM) pooling to produce the final global descriptor $\mathcal{D}$.
Training Strategy
SuperGF is trained using a Listwise Average Precision (AP) loss, which optimizes the ranking of an entire image batch directly. This approach offers more stability than traditional triplet losses by considering the global structure of the embedding space.
For the sparse version, an additional attention decorrelation loss is implemented to ensure that different attention slots capture non-redundant information across the image, maximizing the information density of the global fingerprint.
Experiments and Results
The model was evaluated on standard benchmarks including MSLS, Pitts30k, and the RobotCar Seasons dataset, which includes challenging conditions like night, rain, and snow.
Performance and Efficiency
- Localization Accuracy: On the challenging RobotCar Seasons "Night" sequence, SuperGF (paired with SuperGlue) achieved a pose recall of 34.2 / 70.3 / 93.0 at various distance/angle thresholds. This represents a massive improvement over the standard NetVLAD+SuperPoint baseline (6.6 / 17.1 / 32.2).
- Computational Efficiency: The SuperGF-sparse model is remarkably lightweight, requiring only 2.2 MB of memory and 2.3 ms for feature extraction on an NVIDIA Titan RTX. This is nearly 70x smaller than competing state-of-the-art models like SFRS (149 MB).
- Robustness: SuperGF showed particular strength in cross-condition matching, successfully localizing nighttime queries against daytime databases where traditional semantic-heavy models often fail.
Conclusion
SuperGF demonstrates that local features contain sufficient information for robust global retrieval when aggregated through a global receptive field. By eliminating the need for a separate global backbone, SuperGF provides a faster, more memory-efficient, and more accurate solution for real-time visual localization. Future work may explore end-to-end training of the local feature extractor specifically for this unified task to further optimize performance.
Summary updated