Abstract
Automating the labeling of 3D point cloud data is a critical challenge in the Architecture, Engineering, and Construction (AEC) industry. This paper explores the efficacy of transfer learning to bridge the "domain gap" between finished indoor environments and raw shell construction sites. By pre-training state-of-the-art transformer models (PTv2, PTv3, and SWIN3D) on large-scale datasets like S3DIS and Structured3D, and fine-tuning them on a specialized construction dataset, the authors achieved a mean Intersection over Union (mIoU) of 75%. The study highlights the potential of synthetic data and pre-labeling workflows to significantly reduce the manual effort required for high-resolution as-built documentation.
Introduction
In computer vision, 3D point clouds are essential for capturing the "as-built" status of buildings. However, training models to recognize structural components requires vast amounts of human-annotated data. Currently, a significant domain gap exists: most open-source datasets focus on finished spaces, whereas shell construction sites are characterized by raw concrete, exposed brick, and scaffolding.
This research investigates whether knowledge from finished indoor datasets can be transferred to automatically pre-label raw construction data, thereby accelerating the development of industry-specific AI tools.
Methodology
The researchers employed a three-phase experimental design using Point Transformer V2 (PTv2), Point Transformer V3 (PTv3), and SWIN3D architectures:
- Baseline Training: Models were trained from scratch on a small, custom construction dataset to establish a performance floor.
- Cross-Domain Evaluation: Pre-trained models were tested directly on construction scenes using a translation layer to map labels (e.g., mapping "slab" to "floor").
- Transfer Learning: Backbones pre-trained on large datasets were fine-tuned with construction-specific data to optimize performance.
Dataset Analysis
The study utilized established benchmarks (S3DIS, ScanNet v2, Structured3D, VASAD) alongside a new custom validation dataset. The custom set consists of 36 high-resolution scans from a residential building under construction, captured using a FARO Focus M70 scanner.
As shown in Figure 1, the authors mapped classes across datasets to ensure compatibility. The custom dataset includes 11 classes, ranging from primary structural elements like walls and floors to more complex installations and equipment.
Key Results and Insights
The experiments yielded several critical findings:
- Transfer Learning Efficacy: Fine-tuning pushed the mIoU to approximately 75%, significantly outperforming baseline models trained from scratch.
- Synthetic Data Performance: Pre-training on the synthetic Structured3D dataset produced some of the best results, suggesting that its geometric diversity translates well to real-world construction environments despite being synthetic.
- Geometric Reliability: Models achieved near-perfect accuracy (up to 98% IoU) on planar surfaces like walls and ceilings. However, complex geometries like beams and stairs remain challenging, often being misclassified due to their proximity to major surfaces.
- Architecture Comparison: PTv3 and SWIN3D outperformed PTv2. PTv3 showed strength in identifying door frames and windows, while SWIN3D excelled at segmenting smaller electrical installations.
Limitations and Future Work
Despite the promising 75% mIoU, limitations remain. The current validation set is limited to a single building complex, which may affect generalization to other construction styles. Furthermore, the reliance on RGB and coordinates suggests that incorporating surface normals and laser reflection intensity could improve the segmentation of complex objects. Future work will focus on expanding dataset diversity to address class imbalances and cover a broader spectrum of AEC components.
Conclusion
This research demonstrates that a synergistic multi-dataset approach can effectively pre-label construction point clouds. By achieving high accuracy on core structural elements, these models can serve as powerful pre-labeling tools, allowing human experts to focus on refining complex details and drastically reducing the time required to build robust, industry-ready datasets.
Summary updated