A Multi-View Dense Point Cloud Generation Algorithm Based on Low-Altitude Remote Sensing Images

Zhenfeng Shao, Nan Yang, Xiongwu Xiao, Lei Zhang, Zhe PengRemote 2016 · DOI

TL;DR

This paper introduces a novel algorithm to generate denser and more detailed 3D point clouds from low-altitude remote sensing images. By expanding upon existing Multi-View Stereo (MVS) techniques with a patch-based optimization approach, the method significantly improves point cloud density and accuracy, making it a competitive alternative to laser scanning for large-scale 3D modeling.

What Problem Does This Paper Address?

Generating accurate and dense 3D point clouds is crucial for various applications, from topographic monitoring and cultural heritage preservation to urban planning and environmental modeling. While laser scanning excels at producing high-quality point clouds, it can be expensive and less flexible for large-scale coverage. Image-based 3D reconstruction, particularly using Multi-View Stereo (MVS), offers a more cost-effective and flexible alternative. However, traditional MVS methods often struggle to achieve the desired density and can be susceptible to noise and occlusions, especially when dealing with low-altitude remote sensing imagery. This paper aims to bridge this gap by developing an algorithm that enhances the density and robustness of point clouds generated from such imagery.

Methods

The proposed algorithm builds upon existing MVS techniques, specifically leveraging the output of the Patch-based Multi View Stereo (PMVS) algorithm as a starting point. The core of the method involves a four-step process:

  1. PMVS Point Cloud Generation: This initial step uses PMVS to generate a semi-dense set of "seed" patches. PMVS is chosen for its accuracy and completeness in generating these initial patches.
  2. Patch-Based Point Cloud Expansion: This is a key innovation. The algorithm expands these seed patches by utilizing projection rules between image pixels and the patch's local tangent plane. This effectively grows the point cloud by segmenting image windows and projecting them onto the patch, theoretically increasing density.
  3. Patch-Based Multiphoto Geometrically Constrained Matching (MPGC) Optimization: To refine the expanded point cloud, a modified MPGC algorithm is employed. This method optimizes points on the expanded patches by considering epipolar line constraints, least squares adjustment, and accounting for radiation distortion. This approach leverages redundant measurements from multiple images to mitigate the impact of noise and occlusions.
  4. Outlier Filter: A final step uses a density constraint to remove erroneous points, further improving the overall accuracy and reliability of the generated point cloud.

Datasets

The authors evaluated their method using three distinct datasets of low-altitude remote sensing images. Each dataset comprised five images captured from:

  • Northwestern University (Shaanxi Province, China)
  • Yangjiang (Guangdong Province, China)
  • Hainan (a province in China)

Camera calibration parameters (K-matrix) were obtained from laboratory calibration and bundle adjustment. For aero-triangulation to acquire external orientation elements (C-matrix and R-matrix), commercial low-altitude photogrammetric processing software called GodWork was used. The paper does not introduce new benchmark datasets but rather uses these existing datasets to compare its method against established software.

Key Insights and Ideas

  • Patch Expansion is Key to Density: The core idea of expanding PMVS seed patches by projecting image windows onto the patch plane is a direct strategy to increase point cloud density. This moves beyond simply refining existing points to actively generating new ones.
  • MPGC for Robustness: Integrating a patch-based MPGC algorithm, which incorporates epipolar constraints and least squares adjustment, is crucial for improving accuracy and handling challenging conditions like noise and occlusions. This leverages the strengths of both geometric constraints and statistical optimization.
  • Leveraging PMVS as a Seed: Using PMVS results as a starting point is a pragmatic approach. It capitalizes on the established strengths of PMVS for initial feature detection and patch generation, allowing the proposed method to focus on density enhancement and refinement.
  • Ground Resolution Matters: The study highlights that the ground resolution of the input images significantly impacts the achievable point cloud density. Higher resolution imagery leads to denser and more detailed reconstructions.

Key Results

The proposed method demonstrated significant improvements in point cloud density and detail compared to PMVS, SURE, and Pix4D.

  • Density: The method achieved substantially denser point clouds, with results showing over 40 times more points per square meter than PMVS and over eight times more than Pix4D in some cases. For instance, the Yangjiang dataset yielded a density of 135.122 patches/m².
  • Detail: Qualitative results showed that the proposed method extracted finer details, such as plainer silhouettes, roads, cars, and meticulous roofs, which were less apparent in the outputs of other methods.
  • Accuracy: The accuracy evaluation, measured by relative Euclidean distance, showed average errors of less than 0.5 m for all datasets, indicating reliable performance for 3D reconstruction from low-altitude remote sensing images. The Yangjiang dataset, with the highest ground resolution, achieved the most accurate point cloud (0.166 m average error).
  • Outlier Reduction: The outlier filter effectively reduced the number of erroneous points, with the Yangjiang dataset showing less than four outliers in 10⁵ points.

Limitations and Future Work

The authors acknowledge several limitations and suggest future research directions:

  • Textureless Regions: The method struggles in areas with little or no texture (e.g., flat farmland), as it relies on feature points for expansion. This limits reconstruction quality in such regions.
  • Efficiency: While the method improves density and accuracy, raising the efficiency of image matching is proposed as a future work to extend the method to larger scenes.
  • PMVS Seed Quality: The precision of the image orientation parameters used for the PMVS seed generation directly impacts the final results. High-accuracy calibrated images are recommended.
  • Future Work: The authors suggest improving image matching efficiency for larger-scale reconstructions and enhancing PMVS results in textureless areas.

Summary updated

Abstract

This paper presents a novel multi-view dense point cloud generation algorithm based on low-altitude remote sensing images. The proposed method was designed to be especially effective in enhancing the density of point clouds generated by Multi-View Stereo (MVS) algorithms. To overcome the limitations of MVS and dense matching algorithms, an expanded patch was set up for each point in the point cloud. Then, a patch-based Multiphoto Geometrically Constrained Matching (MPGC) was employed to optimize points on the patch based on least square adjustment, the space geometry relationship, and epipolar line constraint. The major advantages of this approach are twofold: (1) compared with the MVS method, the proposed algorithm can achieve denser three-dimensional (3D) point cloud data; and (2) compared with the epipolar-based dense matching method, the proposed method utilizes redundant measurements to weaken the influence of occlusion and noise on matching results. Comparison studies and experimental results have validated the accuracy of the proposed algorithm in low-altitude remote sensing image dense point cloud generation.
BibTeX
@inproceedings{shao2016multi,
  author = {Zhenfeng Shao and Nan Yang and Xiongwu Xiao and Lei Zhang and Zhe Peng},
  title = {{A Multi-View Dense Point Cloud Generation Algorithm Based on Low-Altitude Remote Sensing Images}},
  booktitle = {Remote},
  year = {2016},
  doi = {10.3390/rs8050381},
}
Figure