Abstract
Local feature matching is a fundamental component of computer vision, essential for tasks such as Simultaneous Localization and Mapping (SLAM) and visual localization. While Vision Transformers (ViTs) have established a new state-of-the-art in matching accuracy, their quadratic computational complexity poses significant challenges for real-time applications. This paper introduces MambaGlue, a novel architecture that integrates the linear-time efficiency of Mamba state-space models with the global context capabilities of Transformers. By employing a parallel MambaAttention mixer and an enhanced confidence regressor, MambaGlue achieves superior accuracy and robustness while maintaining the low latency required for deployment in robotics and augmented reality.
Introduction
Finding corresponding points between image pairs—local feature matching—is a prerequisite for reconstructing 3D scenes and determining camera poses. Recent advancements have been dominated by Transformer-based matchers like SuperGlue and LightGlue. These models leverage self- and cross-attention to reason about the global context of an image, allowing them to resolve ambiguities in textureless regions or under extreme lighting changes.
However, the quadratic scaling of Transformers relative to the number of keypoints creates a computational bottleneck. As the demand for high-resolution matching in real-time systems grows, there is a pressing need for architectures that offer the robustness of Transformers with significantly lower overhead. MambaGlue addresses this by incorporating State Space Models (SSMs), specifically the Mamba architecture, to achieve a more efficient balance between speed and performance.
The MambaGlue Architecture
MambaGlue builds upon the iterative, early-stopping framework introduced by LightGlue, but reimagines the internal processing blocks to optimize for both speed and feature representation.
1. MambaAttention Mixer
The core innovation of MambaGlue is the MambaAttention mixer. Rather than relying solely on standard self-attention, which is computationally expensive, the authors design a block that processes features through three parallel branches:
- Mamba Block: Utilizes selective sequence scanning for efficient, linear-time feature aggregation.
- Standard Self-Attention: Retains the ability to capture complex global dependencies.
- Residual Connection: Ensures stable gradient flow and feature preservation.
By running these branches in parallel without intervening heavy Multi-Layer Perceptrons (MLPs), the model captures both local and global context with a significantly reduced computational footprint compared to pure Transformer architectures.
2. Deep Confidence Score Regressor
To optimize inference speed, modern matchers use "early exit" mechanisms, stopping the iterative process once a confident match is found. While previous models used a simple linear layer for this task, MambaGlue employs a deeper MLP network. This deeper regressor provides a more sophisticated hierarchical understanding of the features. Counterintuitively, adding this complexity makes the model faster in practice; the more accurate confidence predictions allow the model to trigger the early-stopping mechanism more effectively, reducing the total number of iterations required per image pair.
Experimental Results
The model was pre-trained on 5 million image pairs, achieving convergence in just two GPU-days—a significant improvement over previous benchmarks.
Homography and Pose Estimation
On the HPatches dataset, MambaGlue demonstrated exceptional precision in homography estimation. At a 3px threshold, it achieved 94.6% precision, notably outperforming LightGlue (88.9%) and SuperGlue (87.4%).
In outdoor relative pose estimation using MegaDepth-1500, MambaGlue achieved the highest Area Under the Curve (AUC) among sparse matchers. With early stopping enabled, it reached an AUC of 79.9 (@20°) in only 33.1 milliseconds. This represents a superior speed-to-accuracy trade-off, rivaling even dense matchers like LoFTR at coarse error thresholds while remaining much lighter.
Visual Localization
Evaluation on the Aachen Day-Night benchmark confirmed MambaGlue's robustness in varying environmental conditions. It outperformed all baseline sparse matchers in both daytime and nighttime scenarios. Despite the added architectural complexity of the hybrid mixer, the throughput remained competitive at 16.7 pairs per second, compared to LightGlue’s 17.2 pairs per second.
Discussion and Insights
The success of MambaGlue highlights several key insights for future vision architectures:
- Hybrid Efficiency: Pure SSMs can sometimes struggle with global spatial context, while pure Transformers are slow. A parallel hybrid approach provides a "sweet spot" for high-performance computer vision.
- Intelligent Early Exiting: Investing parameters into the decision-making components of a model (like the confidence regressor) can yield net gains in speed by reducing redundant computation.
- Sparse vs. Dense: MambaGlue proves that sparse matching, when enhanced with efficient global reasoning, can compete with dense matching methods that are traditionally much more resource-intensive.
Conclusion and Future Work
MambaGlue represents a significant step toward efficient, robust local feature matching. By successfully integrating Mamba blocks into the matching pipeline, the authors have reduced the computational barriers to high-quality visual localization.
A remaining limitation is the continued reliance on Transformer-based cross-attention for inter-image communication. Future research will focus on developing a "pure Mamba" matcher to entirely eliminate quadratic complexity, further pushing the boundaries of real-time 3D vision.
Summary updated