RECONSTRUCTION OF 3D MODELS FROM POINT CLOUDS WITH HYBRID REPRESENTATION

Pingbo Hu, Zhen Dong, Pengfei Yuan, Fuxun Liang, Bisheng Yang2018 · DOI

Abstract

The three-dimensional (3D) reconstruction of urban buildings from point clouds has long been an active topic in applications related to human activities. However, due to the structures significantly differ in terms of complexity, the task of 3D reconstruction remains a challenging issue especially for the freeform surfaces. In this paper, we present a new reconstruction algorithm which allows the 3D-models of building as a combination of regular structures and irregular surfaces, where the regular structures are parameterized plane primitives and the irregular surfaces are expressed as meshes. The extraction of irregular surfaces starts with an over-segmented method for the unstructured point data, a region growing approach based the adjacent graph of super-voxels is then applied to collapse these super-voxels, and the freeform surfaces can be clustered from the voxels filtered by a thickness threshold. To achieve these regular planar primitives, the remaining voxels with a larger flatness will be further divided into multiscale super-voxels as basic units, and the final segmented planes are enriched and refined in a mutually reinforcing manner under the framework of a global energy optimization. We have implemented the proposed algorithms and mainly tested on two point clouds that differ in point density and urban characteristic, and experimental results on complex building structures illustrated the efficacy of the proposed framework.

Introduction

Reconstructing 3D building models from point clouds—generated via LiDAR or photogrammetry—is essential for urban planning and digital twin development. However, existing methods often struggle with the geometric diversity of modern architecture. Data-driven methods typically assume buildings are purely polyhedral, failing when they encounter curved surfaces. Conversely, model-driven methods rely on predefined shape libraries that cannot account for unique, non-standard designs. This research addresses the gap by proposing a hybrid framework that reconstructs buildings as a combination of parametric plane primitives for regular structures and flexible meshes for irregular, freeform surfaces.

Methodology: The Hybrid Pipeline

The proposed framework utilizes a multi-stage pipeline to transition from raw point data to a structured hybrid model. To manage computational complexity, the algorithm first groups individual points into "super-voxels" based on similar normals and curvatures.

Figure 1: The proposed multi-stage pipeline for hybrid 3D building reconstruction.

As shown in the pipeline above, the process involves three key phases:

  1. Freeform Surface Extraction: The system identifies irregular areas first using a "thickness threshold" (dispersion angle). Voxels that do not fit a flat plane are clustered and set aside to be reconstructed as meshes via Poisson surface reconstruction.
  2. Multi-scale Planar Segmentation: The remaining "flat" voxels are analyzed at multiple scales to identify candidate planes, ensuring both large facades and small architectural details are captured.
  3. Global Energy Optimization: A graph-cut-based optimization refines the planes by balancing data fit, neighbor alignment (smoothness), and "label cost" (minimizing the total number of planes used).

Finally, the meshes and planes are integrated using interpolation at transition regions to ensure a watertight model.

Experimental Evaluation

The framework was evaluated on two distinct datasets:

  • S1 (Airborne Laser Scanning - ALS): A sparse dataset (23 pts/m²) of a building with a prominent dome, testing the algorithm's ability to handle freeform surfaces in low-density environments.
  • S2 (Merged MVS and TLS): A dense but noisy dataset (45 pts/m²) combining Multi-View Stereo and Terrestrial Laser Scanning, serving as a stress test for uneven point distribution.

Key Results and Insights

The hybrid approach successfully produced models where flat walls were represented by clean mathematical planes and decorative or curved elements were represented by detailed meshes. By identifying irregular surfaces before planar optimization, the algorithm avoids the "stair-step" artifacts common when traditional methods try to approximate curves with flat tiles.

Figure 2: Segmentation results for dataset S1, illustrating the successful separation of planar roof sections and the irregular dome structure.

Quantitatively, the average distance from original points to reconstructed planes was 3.8 cm for the ALS data (S1) and 6.2 cm for the noisier MVS/TLS data (S2). The system proved robust in handling complex transition regions where different geometric types meet, a common failure point for automated modeling.

Limitations and Future Work

While effective, the current framework relies on manual tuning for certain thresholds, such as the thickness metric for freeform surfaces. Future research will focus on developing adaptive parameters that adjust automatically to local noise and point density. Additionally, the authors aim to improve the smoothing of "seams" between mesh and planar representations to enhance the aesthetic quality of the final 3D models.

Summary updated

Abstract

The three-dimensional (3D) reconstruction of urban buildings from point clouds has long been an active topic in applications related to human activities. However, due to the structures significantly differ in terms of complexity, the task of 3D reconstruction remains a challenging issue especially for the freeform surfaces. In this paper, we present a new reconstruction algorithm which allows the 3D-models of building as a combination of regular structures and irregular surfaces, where the regular structures are parameterized plane primitives and the irregular surfaces are expressed as meshes. The extraction of irregular surfaces starts with an over-segmented method for the unstructured point data, a region growing approach based the adjacent graph of super-voxels is then applied to collapse these super-voxels, and the freeform surfaces can be clustered from the voxels filtered by a thickness threshold. To achieve these regular planar primitives, the remaining voxels with a larger flatness will be further divided into multiscale super-voxels as basic units, and the final segmented planes are enriched and refined in a mutually reinforcing manner under the framework of a global energy optimization. We have implemented the proposed algorithms and mainly tested on two point clouds that differ in point density and urban characteristic, and experimental results on complex building structures illustrated the efficacy of the proposed framework.
BibTeX
@inproceedings{hu2018reconstruction,
  author = {Pingbo Hu and Zhen Dong and Pengfei Yuan and Fuxun Liang and Bisheng Yang},
  title = {{RECONSTRUCTION OF 3D MODELS FROM POINT CLOUDS WITH HYBRID REPRESENTATION}},
  booktitle = {˜The},
  year = {2018},
  doi = {10.5194/isprs-archives-xlii-2-449-2018},
}
Figure