PHOTOMATCH: AN OPEN-SOURCE MULTI-VIEW AND MULTI-MODAL FEATURE MATCHING TOOL FOR PHOTOGRAMMETRIC APPLICATIONS

Diego González‐Aguilera, Esteban Ruíz de Oña, Luis López-Fernández, Elisa Mariarosaria Farella, E. K. Stathopoulou, I. Toschi, Fabio Remondino, Pablo Rodríguez‐Gonzálvez, David Hernández‐López, Andrea Fusiello, Francesco Nex2020 · DOI

Abstract

Automatic feature matching is a crucial step in Structure-from-Motion (SfM) applications for 3D reconstruction purposes. From an historical perspective we can say now that SIFT was the enabling technology that made SfM a successful and fully automated pipeline. SIFT was the ancestor of a wealth of detector/descriptor methods that are now available. Various research activities have tried to benchmark detector/descriptors operators, but a clear outcome is difficult to be drawn. This paper presents an ISPRS Scientific Initiative aimed at providing the community with an educational open-source tool (called PhotoMatch) for tie point extractions and image matching. Several enhancement and decolorization methods can be initially applied to an image dataset in order to improve the successive feature extraction steps. Then different detector/descriptor combinations are possible, coupled with different matching strategies and quality control metrics. Examples and results show the implemented functionality of PhotoMatch which has also a tutorial for shortly explaining the implemented methods.

Introduction

Feature matching is the "make or break" stage of any Structure-from-Motion (SfM) pipeline. If keypoints are poorly extracted or incorrectly matched, the resulting 3D reconstruction is compromised. While the industry has evolved from the revolutionary SIFT algorithm to a wide array of modern alternatives, researchers often struggle to determine which combination of detectors and descriptors is optimal for specific datasets—particularly those involving challenging multi-modal or rotated imagery.

PhotoMatch addresses this gap by providing an open-source, GPU-accelerated environment to test, combine, and evaluate state-of-the-art feature detectors and descriptors. Unlike many "black-box" commercial tools, PhotoMatch offers transparency and flexibility for benchmarking algorithms under varying geometric and radiometric conditions.

Methodology

PhotoMatch is developed in C++ using the Qt framework and leverages OpenCV for core computer vision functions. The tool implements a structured six-step pipeline designed for both educational and production use.

The modular workflow of PhotoMatch, from image pre-processing to result export.

The pipeline consists of:

  1. Project Definition: Enables side-by-side comparison of different algorithm configurations.
  2. Pre-processing: Includes radiometric enhancement (e.g., RSWHE) and decolorization to improve feature detectability.
  3. Feature Extraction & Description: Supports mixing detectors (SIFT, SURF, MSER, ORB, AKAZE, BRISK) with various descriptors (BOOST, BRIEF, DAISY, FREAK).
  4. Feature Matching: Implements Brute-Force and FLANN strategies, alongside robust filtering like Grid-based Motion Statistics (GMS).
  5. Quality Control: Provides analytical tools such as ROC curves and F-matrix/H-matrix verification.
  6. Export: Results are compatible with major SfM and bundle adjustment software.

Experimental Datasets

The tool was evaluated using three distinct datasets:

  • Aerial Oblique: The ISPRS/EuroSDR Dortmund benchmark, featuring nadir and oblique aerial images.
  • Terrestrial: Five convergent and highly rotated images of the Modena Cathedral to test rotational invariance.
  • Multi-modal: Ten co-registered thermographic and visible-spectrum images.

Results and Discussion

One of the primary insights from the evaluation is that "native" detector-descriptor pairs are not always the most effective. For instance, combining the BRISK detector with a SIFT descriptor outperformed the standard BRISK+BRISK combination in specific scenarios.

PhotoMatch interface showing multi-view pass points (a) and performance evaluation via ROC curves (b).

Key findings include:

  • Dortmund Dataset: SIFT+SIFT yielded the highest match count and best true-positive rate. Conversely, the BRIEF descriptor performed poorly across all tests.
  • Terrestrial Dataset: SURF-SURF was exceptionally resilient to significant tilt and rotation, identifying 539 correct matches compared to 450 for SIFT+SIFT.
  • Multi-modal Matching: Success in matching thermal to visible imagery depended heavily on tuning salience thresholds. Using the MSD (Maximal Self-Dissimilarity) detector with a SIFT descriptor, the authors achieved a matching efficiency of 84.2%.

Conclusion

PhotoMatch serves as a "Swiss Army Knife" for photogrammetric feature matching, offering a modular platform for benchmarking traditional handcrafted features. While future work aims to integrate deep-learning-based descriptors (e.g., SuperPoint), the current tool provides a robust, open-source solution for optimizing 3D reconstruction workflows.

PhotoMatch is available as an open-source project on GitHub.

Summary updated

Abstract

Automatic feature matching is a crucial step in Structure-from-Motion (SfM) applications for 3D reconstruction purposes. From an historical perspective we can say now that SIFT was the enabling technology that made SfM a successful and fully automated pipeline. SIFT was the ancestor of a wealth of detector/descriptor methods that are now available. Various research activities have tried to benchmark detector/descriptors operators, but a clear outcome is difficult to be drawn. This paper presents an ISPRS Scientific Initiative aimed at providing the community with an educational open-source tool (called PhotoMatch) for tie point extractions and image matching. Several enhancement and decolorization methods can be initially applied to an image dataset in order to improve the successive feature extraction steps. Then different detector/descriptor combinations are possible, coupled with different matching strategies and quality control metrics. Examples and results show the implemented functionality of PhotoMatch which has also a tutorial for shortly explaining the implemented methods.
BibTeX
@inproceedings{gonzalezaguilera2020photomatch,
  author = {Diego González‐Aguilera and Esteban Ruíz de Oña and Luis López-Fernández and Elisa Mariarosaria Farella and E. K. Stathopoulou and I. Toschi and Fabio Remondino and Pablo Rodríguez‐Gonzálvez and David Hernández‐López and Andrea Fusiello and Francesco Nex},
  title = {{PHOTOMATCH: AN OPEN-SOURCE MULTI-VIEW AND MULTI-MODAL FEATURE MATCHING TOOL FOR PHOTOGRAMMETRIC APPLICATIONS}},
  booktitle = {˜The},
  year = {2020},
  doi = {10.5194/isprs-archives-xliii-b5-2020-213-2020},
}
Figure