Self-Supervised Large Scale Point Cloud Completion for Archaeological Site Restoration

Aocheng Li, James Zimmer-Dauphinee, Rajesh Kalyanam, Ian Lindsay, Parker VanValkenburgh, Steven A. Wernke, Daniel G. Aliaga

Introduction

Digital archaeology has revolutionized the preservation of cultural heritage through the use of LiDAR and photogrammetry. These technologies allow researchers to create detailed 3D point clouds of ancient sites. However, the resulting data is frequently incomplete due to physical erosion over centuries or occlusions caused by vegetation and structural debris. Restoring these "missing pieces" is critical for both historical analysis and public visualization.

Traditional AI methods for point cloud completion often struggle with archaeological data for three primary reasons. First, they are typically trained on clean, synthetic CAD models (like cars or furniture), which fail to capture the noisy, irregular reality of ruins. Second, supervised learning requires "ground truth"—a perfect version of the object—which does not exist for ancient ruins. Finally, the sheer scale of archaeological sites, often comprising millions of points, makes 3D processing computationally prohibitive.

Methodology

To address these challenges, this research introduces a novel pipeline that shifts the problem from complex 3D space into a specialized 2D image processing framework. This approach allows the system to leverage mature deep learning architectures while maintaining high geometric fidelity.

MCOP Representation

The core of the method is the Multiple-Center-of-Projection (MCOP) representation. Rather than a standard 2D photograph, the 3D point cloud is projected into a 5-channel image that stores RGB color, depth, and rotation data. This is achieved by simulating a virtual camera path around the structure, capturing data "slits" that are then stitched into a coherent 2D map.

Self-Supervised Patch-Based Inpainting

To overcome the lack of ground truth data, the researchers employed a self-supervised learning strategy. Using the LaMa inpainting architecture, the model is trained to observe small, intact patches of a structure to learn its specific masonry patterns and textures. By understanding the local logic of the construction, the model can then "fill in" missing areas—such as the tops of walls that have eroded away—based on the surviving sections at the base.

Multi-Loss Optimization

The training process is guided by a multi-faceted loss function to ensure the reconstruction is both visually and geometrically accurate:

  • Adversarial Loss: Ensures generated patches are indistinguishable from real archaeological surfaces.
  • Regularity Loss: Guarantees that when the 2D image is projected back into 3D, the resulting geometry maintains structural integrity (e.g., correct wall angles).
  • Appearance Consistency: Uses a pretrained VGG19 network to ensure that color and texture remain consistent across the entire restored surface.

Experimental Results

The system was tested on datasets from two major archaeological sites in Peru: Mawchu Llacta and Huamanmarca. To evaluate performance, the researchers created the "Mawchu Incomplete" benchmark, where 20% to 80% of the original scan data was intentionally removed.

Quantitative Performance

The proposed method significantly outperformed existing self-supervised point cloud techniques, such as ACL-SPC and P2C. In terms of Chamfer Distance (a metric for geometric error), the model achieved a score of 8.22, compared to 23.57 for the leading competitor. It also achieved the lowest Fréchet Inception Distance (FID), indicating superior visual and textural realism.

Robustness and Fidelity

The model demonstrated remarkable resilience, successfully restoring structures even when up to 95% of the original surface was missing. Unlike many methods that focus solely on shape, this pipeline restores RGB textures, providing archaeologists with a more complete understanding of the building materials and aesthetic choices of the original builders.

Discussion and Limitations

By reducing the dimensionality of the problem from 3D to 2D MCOP images, the researchers bypassed the memory constraints of 3D convolutions, allowing for the processing of structures with over 15 million points. However, some limitations remain. The system currently requires manual "rough boundary annotations" to define the areas for restoration. Additionally, environmental shadows present during the initial scan can sometimes be misinterpreted as missing data, and there is a practical ceiling on the resolution of MCOP images that can be processed in a single pass.

Conclusion

This self-supervised approach represents a significant advancement in digital restoration. By learning the inherent patterns of ancient masonry from the structures themselves, the system provides a scalable, high-fidelity solution for reconstructing the world's disappearing archaeological heritage without the need for synthetic training data or non-existent ground truths.

Summary updated

BibTeX
@inproceedings{li2025self,
  author = {Aocheng Li and James Zimmer-Dauphinee and Rajesh Kalyanam and Ian Lindsay and Parker VanValkenburgh and Steven A. Wernke and Daniel G. Aliaga},
  title = {{Self-Supervised Large Scale Point Cloud Completion for Archaeological Site Restoration}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.01098},
}
Figure