Automatic Registration of Optical Images with Airborne LiDAR Point Cloud in Urban Scenes Based on Line-Point Similarity Invariant and Extended Collinearity Equations

Shubiao Peng, Hongchao Ma, Liang Zhang2019 · DOI

TL;DR

This paper presents a novel, two-stage approach for precisely aligning aerial optical images with airborne LiDAR point clouds in urban environments. By combining robust line feature extraction and matching with extended collinearity equations, the method achieves higher accuracy and automation than existing techniques, crucial for 3D urban scene analysis.

What Problem Does This Paper Address?

High-resolution optical images are invaluable for urban analysis, but they lack the 3D geometric information that airborne LiDAR provides. Conversely, LiDAR point clouds offer precise 3D data but can be sparse and lack the rich spectral information of optical images. To effectively fuse these datasets for applications like 3D reconstruction, change detection, and urban planning, they must be precisely registered. While georeferencing can provide initial alignment, systematic errors from sensors (GPS, IMU) or calibration issues often lead to misalignments. Traditional image-to-image registration methods struggle with the irregular nature of LiDAR point clouds, and existing LiDAR-to-image registration methods often involve indirect steps or complex mathematical models. This paper aims to bridge this gap by proposing a direct, accurate, and automated registration method specifically for urban scenes.

Methods

The proposed method employs a coarse-to-precise registration strategy:

  • Coarse Registration: This initial step uses a conventional point-feature-based method. Points are extracted from both the optical image and the LiDAR data using a matured point extractor (e.g., Förstner operator). These matched points serve as control information to establish a Direct Linear Transformation (DLT) model, aligning the datasets roughly. If the optical images are already georeferenced, this step is skipped.
  • Precise Registration: This stage is the core contribution.
  • Line Feature Extraction and Refinement: Straight lines, primarily building roof edges, are extracted from both datasets. For LiDAR data, an "infinitesimal feature analysis method" is introduced to refine these edge lines, leveraging the scanning characteristics of the LiDAR system to improve accuracy, especially for building facades and roof planes.
  • Line Matching (Line-Point Similarity Invariant): Matched points from the coarse registration guide the search for matched lines. A "line-point similarity invariant" is proposed. This invariant leverages the property that the ratio of distances from two points to a line remains unchanged under affine transformation. By defining a similarity measure based on these distance ratios and considering matched points within defined search regions, lines are matched robustly, even with potential outliers in the point matches.
  • Extended Collinearity Equations: Once matched lines are established, a transformation function based on "extended collinearity equations" is applied. These equations model the relationship between 2D image pixels and 3D object points (from the LiDAR point cloud) by incorporating line segments as primitives instead of just points. This allows for direct 2D-to-3D registration and the estimation of the camera's exterior orientation elements.

Datasets

The experiments were conducted using an aerial image and an airborne LiDAR point cloud acquired over Changchun, China. The LiDAR data was collected by a Leica ALS70 system with an average point distance of 0.7 meters and accuracies of approximately 0.10–0.15 m (vertical) and 0.2–0.3 m (horizontal). The aerial image was captured by a Leica RCD105 camera with a resolution of 5412 × 7216 pixels and a ground resolution of 0.14 m. No new benchmark datasets were introduced; existing data was used for evaluation.

Key Insights and Ideas

  • Bridging the 2D-3D Gap with Lines: The paper effectively demonstrates that using line features, particularly building edges, can overcome the challenges of registering irregularly spaced LiDAR data with 2D images. Lines provide more stable and abundant features than points in urban environments.
  • Infinitesimal Feature Analysis for LiDAR: The novel "infinitesimal feature analysis method" is a key innovation for refining LiDAR line features. By considering the scanning geometry and projecting echoes onto planes, it significantly improves the accuracy of extracted building edges, which are crucial for the subsequent registration steps.
  • Line-Point Similarity Invariant for Robust Matching: The proposed line-point similarity invariant provides a principled way to match lines by leveraging both point correspondences and geometric properties. This approach is more robust to noise and outliers than methods relying solely on point features or image texture.
  • Direct Registration via Extended Collinearity Equations: By extending the traditional collinearity equations to handle line segments, the paper enables a direct and rigorous mathematical model for 2D-to-3D registration, avoiding intermediate conversions that can introduce errors.

Key Results

The experimental results demonstrate the effectiveness of the proposed method:

  • Improved Accuracy: After coarse registration, the maximum, mean, and RMSE errors between manually selected check lines were 1.52 m, 0.61 m, and 0.45 m, respectively. Following precise registration using the proposed method, these errors significantly decreased to 0.52 m, 0.24 m, and 0.13 m, respectively.
  • High Automation: The method is designed for automatic registration, relying on feature extraction and matching algorithms.
  • Visual Quality: 3D visualizations of the registered data show a high quality of alignment, with buildings and trees clearly matching between the LiDAR point cloud and the optical image. The paper notes that even with low image resolution and point density, the registration quality is readily ascertained.

Limitations and Future Work

  • Reliance on Line Features: The method's accuracy is heavily dependent on the accurate extraction of line segments, particularly from the LiDAR point cloud, which serves as the master dataset. In areas lacking distinct line segments (e.g., mountains, open terrain), the method might be less effective.
  • Extension to Curves: The authors acknowledge that in areas lacking line segments, curves could potentially mimic linear features. However, extending the collinearity equations to handle curves remains a technical challenge for future research.
  • Image Resolution vs. Point Density: The paper notes that the interplay between image resolution and LiDAR point density significantly impacts feature extraction and, consequently, registration accuracy. Determining the optimal balance for registration requires extensive experimentation.
  • Computational Cost: While the method aims for efficiency, the paper mentions that some existing methods for feature extraction can be computationally expensive. The proposed approach seeks to mitigate this by combining point and line features effectively.

Summary updated

BibTeX
@misc{peng2019automatic,
  author = {Shubiao Peng and Hongchao Ma and Liang Zhang},
  title = {{Automatic Registration of Optical Images with Airborne LiDAR Point Cloud in Urban Scenes Based on Line-Point Similarity Invariant and Extended Collinearity Equations}},
  year = {2019},
  doi = {10.3390/s19051086},
}
Figure