Abstract
Large-scale 3D urban reconstruction from satellite imagery often suffers from low-quality building facades due to oblique viewing angles, occlusions, and low resolution. This paper presents an inverse procedural modeling pipeline that reconstructs high-quality, crisp facades by inferring architectural grammars from noisy data. By leveraging structural regularity as a prior, the system synthesizes complete facades that are geometrically consistent and resolution-independent.
Introduction
While satellite imagery has revolutionized 3D urban mapping, capturing building facades remains a significant challenge. Facades in these images are typically low-resolution (often only 20 pixels tall), blurred, and frequently obscured by shadows, vegetation, or adjacent structures. Traditional computer vision methods, such as semantic segmentation or image in-painting, often fail to maintain architectural integrity when faced with such sparse and noisy input. This research addresses the problem by moving beyond simple image-based reconstruction toward a procedural approach that "guesses the rules" of a building's design.
Methodology: Inverse Procedural Modeling
The proposed system utilizes Inverse Procedural Modeling (IPM) to reconstruct facades. Instead of attempting to enhance the noisy pixels directly, the pipeline identifies the underlying architectural grammar that likely generated the building.
The Pipeline
- Selection & Rejection: A ResNet-based model filters out excessively noisy or occluded views, selecting only the most informative satellite fragments.
- Chip Extraction: The system identifies a "chip"—the cleanest representative fragment of the facade—to avoid distractions from local occlusions like trees.
- Segmentation & Grammar Classification: A Pix2Pix GAN segments the chip into functional classes (windows, doors, walls). The facade is then classified into one of six predefined architectural grammars (e.g., grid layouts).
- Parameter Estimation & Optimization: Parameters such as floor count and window spacing are estimated using ResNet models and further refined via Monte Carlo stochastic optimization to maximize the fit with the initial segmentation.
Regularity as a Prior
The core insight is that urban architecture is inherently regular. By identifying a pattern in a small, clear section of a building, the system can plausibly complete the entire structure, even where data is missing. This procedural output is resolution-independent, allowing for high-quality visualization at any zoom level.
Datasets and Training
To train the models, the authors introduced a new satellite facade dataset featuring 400 rectified images from six US cities, complete with manual annotations for ground truth. To overcome the scarcity of labeled real-world data, the researchers synthesized over 700,000 artificial facades to pre-train the parameter estimation networks across diverse architectural styles.
Key Results
The automated pipeline demonstrates significant improvements over existing image completion methods like DeepFill and PICNet.
- Accuracy: The method achieved an average accuracy of 85.4% across various metrics.
- Optimization Impact: The Monte Carlo optimization step provided a 16% boost in reconstruction quality.
- Scalability: The system is highly efficient, processing approximately 14 buildings per minute, making it suitable for city-scale reconstruction.
Qualitatively, the system produces "crisp" architectural renders that avoid the "smudged" appearance of traditional projective mapping.
Limitations and Future Work
Currently, the system is constrained to six architectural grammars, which may not capture highly unique or non-rectangular designs. Future research aims to expand this grammar set to include finer details like balconies and ledges, as well as incorporating sophisticated procedural textures to simulate realistic materials and weathering.
Summary updated