Abstract
Airborne laser scanning and photogrammetry are two main techniques to obtain 3D data representing the object surface. Due to the high cost of laser scanning, we want to explore the potential of using point clouds derived by dense image matching (DIM), as effective alternatives to laser scanning data. We present a framework to evaluate point clouds from dense image matching and derived Digital Surface Models (DSM) based on automatically extracted sample patches. Dense matching error and noise level are evaluated quantitatively at both the local level and whole block level. Experiments show that the optimal vertical accuracy achieved by dense matching is as follows: the mean offset to the reference data is 0.1 Ground Sampling Distance (GSD); the maximum offset goes up to 1.0 GSD. When additional oblique images are used in dense matching, the mean deviation, the variation of mean deviation and the level of random noise all get improved. We also detect a bias between the point cloud and DSM from a single photogrammetric workflow. This framework also allows to reveal inhomogeneity in the distribution of the dense matching errors due to over-fitted BBA network. Meanwhile, suggestions are given on the photogrammetric quality control.
Introduction
Airborne Laser Scanning (ALS) is widely considered the gold standard for high-precision 3D terrain mapping. However, its high operational costs have led researchers to investigate Dense Image Matching (DIM) as a more economical alternative. DIM utilizes standard aerial photography to generate 3D point clouds. While cost-effective, assessing the quality of DIM data is challenging. Traditional evaluation methods, such as comparing a limited number of GPS ground control points or performing point-to-point comparisons, often fail to capture the spatial distribution of errors or are overly sensitive to noise and outliers. This paper proposes a robust, patch-based framework for large-scale quality assessment.
Methodology
The core of the proposed method is the automatic extraction of planar "patches" from a reference ALS dataset to serve as ground truth for DIM evaluation. The process involves several key stages:
- Patch Detection: Using 3D Hough transforms and surface-growing algorithms, the system identifies perfectly planar surfaces (e.g., 2m x 2m squares) on smooth ground features like roads or plazas.
- Rigorous Screening: Patches are filtered to exclude areas affected by shadows, vegetation (detected via a "Green Index"), or temporal changes in terrain between the ALS scan and the image capture.
- Statistical Analysis: For each patch, metrics such as "Mean Deviation" (measuring accuracy/bias) and "Standard Deviation" (measuring precision/noise) are calculated and aggregated to assess the entire mapping block.
Results and Discussion
The study utilized a dataset of 510 aerial images (nadir and oblique) of Enschede, Netherlands, compared against high-precision ALS data.
Vertical Accuracy and Noise
Experimental results indicate that DIM can achieve a vertical accuracy of approximately 0.1 Ground Sampling Distance (GSD), with maximum offsets reaching 1.0 GSD. A significant finding was the benefit of oblique imagery; incorporating angled views stabilized the block geometry, reducing random noise and improving mean deviation by approximately 1.4 cm compared to nadir-only datasets.
The Overfitting Phenomenon
A critical insight from this research is the risk of "overfitting" during Bundle Block Adjustment (BBA). When engineers apply excessive weight to Ground Control Points (GCPs) to minimize reported errors, the mathematical network can become too rigid. This research detected inhomogeneous error distributions—spatial patterns of inaccuracy—that standard Root Mean Square Error (RMSE) reports failed to reveal. Additionally, a systematic vertical bias of roughly 3 cm was found between raw point clouds and the interpolated Digital Surface Models (DSM) derived from them.
Conclusion
The patch-based evaluation framework provides a rigorous means of detecting systematic errors and noise levels in photogrammetric workflows. While the study successfully demonstrated the potential of DIM as an ALS alternative, it also highlighted the need for careful quality control in BBA to avoid overfitting. Future research will aim to extend this evaluation to more complex vertical structures and diverse vegetation types, while also addressing the temporal gaps between reference and test data captures.
Summary updated