Building Facade Reconstruction by Fusing Terrestrial Laser Points and Images

Shi Pu, George Vosselman2009 · DOI

TL;DR

This paper presents a novel approach to creating detailed 3D building facade models by intelligently fusing data from terrestrial laser scanners and close-range images. By combining the geometric precision of laser scans with the rich visual information of images, the method achieves more accurate and photorealistic reconstructions than either data source alone.

What Problem Does This Paper Address?

Creating accurate 3D models of building facades is crucial for applications like urban planning, heritage preservation, and virtual environments. Traditional manual modeling is time-consuming and prone to inaccuracies. While image-based reconstruction offers visual detail, it struggles with automated 3D structure extraction due to challenges in image interpretation and 3D spatial transformation. Terrestrial laser scanning provides dense 3D point clouds, excellent for geometric detail, but lacks color information, making texturing impossible without additional data. This paper addresses the challenge of effectively integrating these complementary data sources to overcome their individual limitations and achieve automated, accurate, and photorealistic building facade reconstruction.

Methods

The proposed method employs a multi-stage approach:

  • Preprocessing: Exterior orientations of images are calculated using semi-automated methods.
  • Initial Model Generation: The general structure of the building facade is established using planar features extracted from terrestrial laser point clouds, resulting in an initial polyhedron model. This leverages the strengths of laser data for geometric primitives.
  • Geometry Refinement: Significant line features are extracted from images using Canny edge detection and Hough transformation. These image-derived lines are then compared with the edges of the initial laser-based model. Discrepancies are used to refine the model's geometry, improving accuracy, especially for features like windows and wall outlines that are difficult to precisely model from laser data alone.
  • Optimal Texture Selection: For texturing, the method selects the most suitable image for each part of the model. This involves considering the acquisition position relative to the model part and the viewing direction to ensure optimal resolution and visibility, minimizing distortion and occlusion artifacts.
  • Occlusion Removal: A strategy is implemented to handle occlusions (e.g., trees, balconies) in the textured model. If occlusion positions are known, the occluded regions are re-textured using alternative views or predefined textures.

Datasets

Not discussed in the paper. The paper describes a method and demonstrates its applicability with test cases, but does not explicitly list or introduce specific datasets for training or evaluation.

Key Insights and Ideas

  • Complementarity of Data: The core insight is that laser data and image data are highly complementary. Laser scans excel at capturing 3D geometry (planes, lines), while images provide high-resolution visual detail and are better for extracting fine linear features.
  • Image-Guided Geometry Refinement: Using image-derived lines to refine a laser-based geometric model is a powerful technique. This addresses limitations in laser data (e.g., sparse points on windows, curved surfaces) by leveraging the superior edge detection capabilities of image processing.
  • Optimized Texturing for Realism: Simply applying any available texture is insufficient. Selecting textures based on viewing angle, distance, and visibility is crucial for achieving photorealistic results and avoiding artifacts like color contrast issues near edges.
  • Semantic Segmentation Benefits: Associating semantic types (wall, roof, window) with model parts allows for more intelligent texturing and visualization strategies.

Key Results

The paper demonstrates the successful generation of photorealistic 3D building facade models through the described fusion approach. Qualitative results are presented in figures showing building facades before and after refinement and texturing, highlighting significant improvements in geometric accuracy and visual realism. The method effectively refines outlines and textures, addressing issues like poor texture quality due to geometric inaccuracies and occlusions.

Limitations and Future Work

  • Manual Tie-Point Selection: The initial registration of laser points and images relies on manual selection of tie pairs, which is time-consuming. The authors note that automated methods are difficult to achieve reliably due to differences in sensor characteristics.
  • Curved Surfaces: The method struggles with significant curvature in facades, as demonstrated by the "Three Joined Houses" example where a slight curvature led to texturing errors.
  • Strict Hough Transform Thresholds: In some cases, strict thresholds for Hough line detection can prevent the extraction of necessary lines for matching, leading to incomplete refinement.
  • Large Buildings: The authors anticipate that reconstructing large buildings will require more images, and accumulated errors in homography estimation could become a significant issue.
  • Future Work: The paper suggests that additional manual assistance might be needed to ensure accurate exterior orientations for all images. Further examination of the required minimum distance between manually processed images is also proposed. The authors also imply that improving automated feature matching and handling complex geometries like significant curvature are areas for future research.

Summary updated

BibTeX
@misc{pu2009building,
  author = {Shi Pu and George Vosselman},
  title = {{Building Facade Reconstruction by Fusing Terrestrial Laser Points and Images}},
  year = {2009},
  doi = {10.3390/s90604525},
}
Figure