Abstract
Local feature matching is a fundamental task in computer vision, yet most state-of-the-art matchers rely exclusively on 2D image intensity and texture. This paper introduces the Curvature Similarity Extractor (CSE), a plug-in module designed to incorporate 3D surface geometry into the matching process. By estimating local surface curvature from monocular depth maps, the CSE provides a viewpoint-invariant geometric signature that helps reject inconsistent matches. When integrated with popular matchers like SuperGlue and LoFTR, the proposed method achieves state-of-the-art performance across multiple benchmarks.
Introduction
Finding corresponding physical points across different images is essential for 3D reconstruction (SfM) and autonomous navigation (SLAM). While deep learning has significantly improved matching robustness, standard methods often struggle with repetitive patterns or surfaces that appear similar in 2D but differ in 3D structure. For example, a flat wall and a flat floor might share similar textures, leading to false positives in a purely 2D matching framework.
The authors propose that surface curvature—an intrinsic property of the 3D world—can serve as a powerful filter. Because curvature is invariant to camera viewpoint, it allows the system to verify if two points belong to the same physical structure based on their local shape rather than just their appearance.
Methodology
The Curvature Similarity Extractor Pipeline
The core contribution of this work is the Curvature Similarity Extractor (CSE), which can be integrated into existing detector-based or detector-free matching architectures. The pipeline begins by predicting a monocular depth map from the input images using a pre-trained model like MiDaS.
Geometric Feature Extraction
Once the depth map is obtained, the system performs the following steps to extract geometric signatures:
- Quadric Fitting: For each feature point, the system identifies a small local patch in the 3D depth map. It then fits a quadric surface—specifically an ellipsoid—to this local point cloud.
- Curvature Extraction: From the fitted ellipsoid, the system calculates principal curvatures or radii along its axes. These values are inherently 3D and remain constant regardless of camera rotation or translation.
- Similarity Scoring: A similarity function compares the curvature of a point in the first image with candidate points in the second. Matches between points with significantly different geometric profiles are penalized.
Integration and Training
The resulting curvature similarity score is linearly combined with the matching probabilities generated by standard 2D matchers. Crucially, the entire pipeline is differentiable. This allows for end-to-end fine-tuning of the depth prediction network, encouraging it to produce depth maps that prioritize the geometric details most useful for matching rather than just visual accuracy.
Experimental Results
The authors evaluated the CSE module on several major benchmarks, including ScanNet (indoor), MegaDepth (outdoor), and the Aachen Day-Night localization challenge.
Key findings include:
- Performance Gains: The addition of CSE consistently improved the Area Under the Curve (AUC) for pose estimation. On ScanNet, AUC@5° improved by 1.0–1.5%, while MegaDepth saw gains of up to 2.5%.
- Robustness: The method proved particularly effective at eliminating incorrect matches in areas with repetitive textures but distinct geometric profiles.
- Versatility: As a plug-and-play module, CSE successfully boosted the performance of various baseline matchers, including LoFTR and QuadTree.
Conclusion and Future Work
By leveraging 3D surface curvature, the CSE module addresses a significant blind spot in modern 2D feature matchers. While the approach introduces some computational overhead due to depth prediction and quadric fitting, it provides a robust geometric constraint that significantly improves matching accuracy. Future research may focus on optimizing the efficiency of the depth prediction stage and improving performance near object edges where depth discontinuities can complicate curvature estimation.
Summary updated