Transformation Model with Constraints for High-Accuracy of 2D-3D Building Registration in Aerial Imagery

Guoqing Zhou, Qingli Luo, Wenhan Xie, Tao Yue, Jingjin Huang, Yuzhong ShenRemote 2016 · DOI

TL;DR

This paper introduces a novel transformation model for registering 2D aerial images with 3D building models, significantly improving accuracy by incorporating geometric constraints derived from building structures. This approach reduces the reliance on numerous ground control points, making it more practical for urban environments with tall buildings.

What Problem Does This Paper Address?

Accurately aligning 2D aerial images with 3D building models (2D-3D registration) is crucial for tasks like creating realistic urban environments. A major challenge in urban areas, especially with tall buildings, is obtaining a sufficient number of well-distributed ground control points (GCPs) to ensure accurate registration. Traditional methods struggle with this, leading to significant registration errors, particularly when image distortions are uneven across the scene. This unevenness is exacerbated by tall buildings, where different parts of the same building can be affected by varying distortions.

Methods

The core contribution is a "rigorous transformation model" that enhances the conventional photogrammetric collinearity model. The novelty lies in the integration of two geometric constraints derived directly from building structures:

  • Co-planar Condition: This constraint ensures that specific points and lines in 3D space, when projected onto the image, remain co-planar with the camera's perspective center. This leverages the geometric relationships of building features.
  • Perpendicular Condition: This constraint enforces that if two lines are perpendicular in the 3D object space (e.g., edges of a building roof), their projections in the 2D image space should also maintain a perpendicular relationship. This is important because perspective projection doesn't inherently preserve angles.

These constraints are mathematically formulated and combined with the traditional transformation model, which also accounts for image distortions like radial and decentering effects. The model is solved using a least-squares estimation approach.

Datasets

The study uses data from downtown Denver, Colorado, including:

  • Aerial Images: Six aerial images acquired with a Leica RC30 aerial camera, featuring a 65% end lap and 30% side lap.
  • Digital Surface Model (DSM): A DSM with a 0.6m grid spacing, acquired via LiDAR, providing 3D building information.
  • 3D Building Models: Constructed using Constructive Solid Geometry (CSG) based on the DSM, representing buildings as combinations of 3D primitives (cubes, cylinders, etc.) with defined attributes.

The paper also details a method for measuring "GCPs" and checkpoints by using building roof/bottom corners from the 3D model and automatically measuring their corresponding 2D image coordinates, achieving sub-pixel accuracy.

Key Insights and Ideas

  • Geometric Constraints as a Solution: The paper demonstrates that leveraging inherent geometric properties of buildings (co-planarity and perpendicularity) can significantly improve registration accuracy, especially in challenging urban environments.
  • Reduced GCP Dependency: By incorporating these structural constraints, the method achieves high accuracy with substantially fewer GCPs compared to traditional approaches. This is a critical practical advantage.
  • Addressing Asymmetric Distortions: The proposed constraints help mitigate the negative impact of uneven image distortions, which are common in aerial imagery and problematic for tall structures.
  • Synergy of Features: The model effectively combines point features (corners) and linear features (edges) for registration, making it more robust.

Key Results

The experimental results highlight the superiority of the proposed method:

  • Theoretical Accuracy: The proposed model achieved a theoretical accuracy of 0.47 pixels for registration parameters, significantly better than conventional methods which reached 1.23 and 1.09 pixels with more GCPs.
  • 2D Space Accuracy (RMS Error): The Root Mean Square (RMS) error in the 2D image plane for the proposed method was approximately 2 pixels (both x and y directions). This is a dramatic improvement over conventional methods, which had RMS errors around 10 pixels.
  • GCP Efficiency: The proposed method achieved these high accuracies using only 32 points + constraints, whereas conventional methods required 211 GCPs to get close to the performance of the proposed method with fewer points.

Limitations and Future Work

  • Limitations: The paper acknowledges that the conventional registration model is not robust for high-rise buildings in urban areas, which is precisely the problem their method aims to solve. While the proposed method significantly improves accuracy, the paper doesn't explicitly detail limitations of the proposed method itself, beyond the general challenges of image distortion. The reliance on well-defined building structures for constraints might be a limitation in areas with less structured or more complex urban forms.
  • Future Work: The paper focuses on the development and validation of the model. Future work could explore applying this method to different urban datasets, investigating its performance with varying building types and complexities, and potentially integrating it with other advanced registration techniques or sensor data.

Summary updated

Abstract

This paper proposes a novel rigorous transformation model for 2D-3D registration to address the difficult problem of obtaining a sufficient number of well-distributed ground control points (GCPs) in urban areas with tall buildings. The proposed model applies two types of geometric constraints, co-planarity and perpendicularity, to the conventional photogrammetric collinearity model. Both types of geometric information are directly obtained from geometric building structures, with which the geometric constraints are automatically created and combined into the conventional transformation model. A test field located in downtown Denver, Colorado, is used to evaluate the accuracy and reliability of the proposed method. The comparison analysis of the accuracy achieved by the proposed method and the conventional method is conducted. Experimental results demonstrated that: (1) the theoretical accuracy of the solved registration parameters can reach 0.47 pixels, whereas the other methods reach only 1.23 and 1.09 pixels; (2) the RMS values of 2D-3D registration achieved by the proposed model are only two pixels along the x and y directions, much smaller than the RMS values of the conventional model, which are approximately 10 pixels along the x and y directions. These results demonstrate that the proposed method is able to significantly improve the accuracy of 2D-3D registration with much fewer GCPs in urban areas with tall buildings.
BibTeX
@inproceedings{zhou2016transformation,
  author = {Guoqing Zhou and Qingli Luo and Wenhan Xie and Tao Yue and Jingjin Huang and Yuzhong Shen},
  title = {{Transformation Model with Constraints for High-Accuracy of 2D-3D Building Registration in Aerial Imagery}},
  booktitle = {Remote},
  year = {2016},
  doi = {10.3390/rs8060507},
}

Loading related content...

Figure