DINO-RotateMatch: A Rotation-Aware Deep Framework for Robust Image Matching in Large-Scale 3D Reconstruction

Kaichen Zhang, Tianxiang Sheng, Xuanming ShiarXiv 2025 · DOI · arXiv

Abstract

3D reconstruction from unorganized Internet imagery presents significant challenges due to extreme variations in illumination, resolution, and camera orientation. We present DINO-RotateMatch, a framework that integrates DINOv2-based semantic image pairing with a rotation-aware keypoint extraction and matching strategy. By leveraging self-supervised Vision Transformers for scalable pairing and augmenting the matching process with four-way image rotations, our approach significantly improves robustness against viewpoint changes. Evaluated on the Kaggle Image Matching Challenge 2025 dataset, DINO-RotateMatch achieved a Silver Award, demonstrating substantial improvements in mean Average Accuracy (mAA) over standard deep-learning baselines.

Introduction

Building accurate 3D models from "in-the-wild" web-sourced photos is a cornerstone of modern computer vision, yet it remains computationally expensive and sensitive to geometric noise. Two primary bottlenecks hinder current systems: scalability and rotation sensitivity. Exhaustive matching of thousands of images is $O(N^2)$, making it impractical for large datasets. Furthermore, while modern neural matchers like LightGlue offer high precision, they often struggle when images exhibit significant relative rotations—a common occurrence in unstructured photo collections.

Methodology

The DINO-RotateMatch framework addresses these issues through a multi-stage pipeline designed for both efficiency and geometric robustness.

The DINO-RotateMatch pipeline for large-scale 3D reconstruction.

Adaptive Image Pairing

To manage scalability, the framework utilizes DINOv2, a self-supervised Vision Transformer, to generate 768-dimensional global descriptors for each image. For datasets exceeding 20 images, the system performs an adaptive search, attempting to match only the top-20 most similar pairs based on Euclidean distance. This "zero-shot" semantic salience allows the system to focus computational resources on image pairs that likely share overlapping content.

Rotation-Aware Keypoint Extraction and Matching

Standard keypoint detectors often possess hidden orientation biases. DINO-RotateMatch mitigates this by applying the ALIKED detector to four rotated versions of each image ($0^\circ, 90^\circ, 180^\circ, 270^\circ$).

Visualization of rotation detection and keypoint matching across four orientations.

Following extraction, a two-stage matching process is conducted using LightGlue. The system matches the original Image A against all four rotated versions of Image B. A pair is validated if the total number of matched keypoints across these orientations exceeds a threshold of 25. This strategy ensures that features which might be weak or undetected in the original orientation are successfully captured through augmentation.

Experimental Results

The framework was rigorously tested on the Kaggle Image Matching Challenge 2025 dataset, which includes difficult scenarios such as low-texture drone imagery, forested scenes, and nighttime captures.

The performance was measured using mean Average Accuracy (mAA). As shown in the table below, the introduction of the rotation strategy provided a significant performance boost.

Configuration Private mAA Public mAA
Exhaustive + ALIKED + LightGlue (Baseline) 25.00 30.39
DINOv2 Pairing + ALIKED + LightGlue 29.36 33.12
DINO-RotateMatch (Adaptive + Rotation) 37.47 35.18

The results indicate that while DINO-based pairing improves efficiency and baseline accuracy, the rotation-aware strategy is the primary driver for robustness in "in-the-wild" scenarios, increasing the private mAA by over 8 points compared to the pairing-only approach.

Discussion and Future Work

While DINO-RotateMatch offers superior accuracy, it introduces additional computational overhead by running the matcher multiple times per image pair. Future research will focus on optimizing this process, perhaps through more granular learned orientation normalization or early-exit strategies during the matching phase. Additionally, integrating multi-modal data (e.g., depth or infrared) could further enhance performance in extreme environments like dense forests or total darkness.

Summary updated

BibTeX
@misc{zhang2025dino,
  author = {Kaichen Zhang and Tianxiang Sheng and Xuanming Shi},
  title = {{DINO-RotateMatch: A Rotation-Aware Deep Framework for Robust Image Matching in Large-Scale 3D Reconstruction}},
  year = {2025},
  eprint = {2512.03715},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.03715},
}
Figure