Uncertainty Aware Interest Point Detection and Description

Jingbo Zeng, Zaiwang Gu, Weide Liu, Lile Cai, Jun ChengWACV 2025 · DOI

Introduction

Interest point detection—identifying distinctive "landmarks" in an image—is the bedrock of computer vision tasks like SLAM, 3D reconstruction, and autonomous navigation. While deep learning has pushed the boundaries of accuracy, most modern detectors are deterministic and overconfident. A standard neural network will often predict a keypoint in a blurry, low-texture, or highly repetitive region without any indication that the prediction might be unreliable.

In safety-critical applications, this lack of "calibrated uncertainty" can lead to catastrophic matching errors. UAPoint addresses this by incorporating evidential learning to quantify both aleatoric (data-driven) and epistemic (model-driven) uncertainty, providing a confidence score for every detected keypoint.

The UAPoint Framework

UAPoint consists of three main components designed to improve both the reliability of keypoints and the discriminative power of descriptors:

  1. Uncertainty-Aware Keypoint Decoder: Instead of a simple probability heatmap, this branch leverages evidential learning. It treats the heatmap prediction as a Normal-Inverse-Gamma distribution, allowing the model to output parameters ($\tau, \phi, \omega$) to calculate aleatoric and epistemic uncertainty.
  2. Constrained Descriptor Decoder: To improve descriptor quality, the authors propose a constrained sampling scheme. During training, the model selects "hard negatives"—spatially close and visually similar keypoints—forcing the descriptor to learn finer details.
  3. Robust Backbone: The system uses a pretrained ResNet-34 as a feature encoder, providing a more robust representation than the VGG-style backbones used in older models.
The UAPoint architecture includes a shared feature encoder followed by specialized decoders for uncertainty-aware keypoint detection and constrained descriptor learning.

Uncertainty as a Quality Filter

A key insight of UAPoint is that learned uncertainty serves as a highly effective quality filter. By discarding the top 10% most uncertain points, homography estimation accuracy increases, proving the model's self-awareness is physically meaningful.

Qualitatively, UAPoint is significantly better than previous methods like SuperPoint at ignoring repetitive patterns or low-texture regions (such as flat rooftops) that typically produce noisy keypoints. Epistemic uncertainty, in particular, shows a strong correlation with localization error, spiking when the model encounters scenes unlike its training data.

Qualitative comparison showing UAPoint's ability to suppress keypoints in textureless regions (red box) compared to SuperPoint, alongside visualizations of the two types of uncertainty.

Experimental Results

The model was evaluated on several industry-standard benchmarks:

  • HPatches: UAPoint achieved the highest repeatability (0.636 at $\epsilon=1$) and mean Average Precision (mAP) compared to heavyweights like SuperPoint, R2D2, and DISK. It excels at low error thresholds, indicating superior pixel-level precision.
  • Aachen Day-Night: In outdoor visual localization under extreme illumination changes, UAPoint (combined with the LightGlue matcher) outperformed existing state-of-the-art pipelines.
  • Efficiency: Despite the added uncertainty branches, the model remains efficient with an inference time of approximately 5.1ms per image.

Conclusion and Future Work

UAPoint demonstrates that incorporating uncertainty into interest point detection not only improves robustness in challenging environments but also provides a reliable metric for filtering low-quality features. While the ResNet-34 backbone makes it slightly heavier than some mobile-first detectors, the gains in precision and reliability are substantial. Future work may involve integrating these uncertainty metrics directly into the matching cost functions of transformer-based matchers or using them to guide active learning for more diverse training data.

Summary updated

BibTeX
@inproceedings{zeng2025uncertainty,
  author = {Jingbo Zeng and Zaiwang Gu and Weide Liu and Lile Cai and Jun Cheng},
  title = {{Uncertainty Aware Interest Point Detection and Description}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
  doi = {10.1109/wacv61041.2025.00215},
}
Figure