Abstract
Thanks to the recent advances in computational photography and remote sensing, point clouds of buildings are becoming increasingly available, yet their processing poses various challenges. In our work, we tackle the problem of point cloud completion and editing and we approach it via inverse procedural modeling. Contrary to the previous work, our approach operates directly on the point cloud without an intermediate triangulation. Our approach consists of 1) semi-automatic segmentation of the input point cloud with segment comparison and template matching to detect repeating structures, 2) a consensus-based voting schema and a pattern extraction algorithm to discover completed terminal geometry and their patterns of usage, all encoded into a context-free grammar, and 3) an interactive editing tool where the user can create new point clouds by using procedural copy and paste operations, and smart resizing. We demonstrate our approach on editing of building models with up to 1.8M points.
Introduction
Point clouds generated from LiDAR or photogrammetry are essentially unstructured "soups" of millions of points. While they capture high detail, they lack semantic structure—computers cannot distinguish between points belonging to a window, a roof, or a door. This lack of structure makes editing difficult; simple scaling operations often lead to gaps or distorted geometry. Furthermore, real-world scans are frequently incomplete due to occlusions. Traditional solutions involve converting points to meshes (triangulation), which often results in "melted" geometry and loss of fine detail. This work proposes a method to edit buildings while remaining entirely within the point cloud domain, preserving original sampling quality.
Methodology
The proposed pipeline utilizes Inverse Procedural Modeling to transform raw data into a structured, editable grammar.
- Semi-Automatic Segmentation: Users provide rough strokes to guide the system. It employs RANSAC and Euclidean clustering to partition the building into semantic parts like facades and roofs.
- Consensus Models (CM): By identifying repetitive architectural elements (e.g., windows), the system aligns multiple instances using Iterative Closest Point (ICP) and merges them. This "heals" the data, using points from well-scanned areas to fill holes in occluded regions.
- Grammar Discovery: Segments are organized into a hierarchical tree. The system detects spatial patterns and encodes them as rules in a context-free grammar.
- Interactive Editing: The rule-based definition allows for "smart" operations. For example, "smart resizing" adds new windows instead of stretching existing ones, and "procedural copy-paste" enables style transfers between buildings.
Results and Insights
The researchers evaluated their approach on several datasets, including the high-density Lans-le-Villard (1.2M points) and the Hannover urban benchmark.
- Model Completion: The Consensus Model approach increased point counts by 1.2x to 4x, successfully filling gaps.
- Efficiency: Pre-processing takes 1-5 minutes, while individual editing operations are near-instant (1 second to 1 minute) for models up to 1.8 million points.
- Robustness: The algorithm remains effective with up to 20% point removal or 1% coordinate noise.
- Data Redundancy: A key insight is treating architectural repetition as a strength. Averaging multiple noisy instances produces a result superior to any single scan.
Conclusion
While the system currently requires some user input for segmentation and struggles with non-linear architectural patterns, it demonstrates a powerful shift from "dumb" point clouds to "smart" editable models. Future work aims to automate rule discovery using machine learning and scale the approach to city-wide LiDAR datasets.
Summary updated