Unsupervised Segmentation for Terracotta Warrior with Seed-Region-Growing CNN(SRG-Net)

Yao Hu, Guohua Geng, Kang Li, Wei Zhou, Xingxing Hao, Xin Cao

TL;DR

This paper introduces SRG-Net, an unsupervised method for segmenting 3D point clouds of terracotta warrior fragments. By combining a novel seed-region-growing algorithm with a dynamic graph convolutional neural network, SRG-Net achieves state-of-the-art accuracy and significantly reduces processing time, aiding archaeologists in restoration efforts.

What Problem Does This Paper Address?

The restoration of ancient artifacts, particularly the vast number of terracotta warrior fragments unearthed from Emperor Qinshihuang's Mausoleum, is a labor-intensive and time-consuming process. Archaeologists manually segment and match these fragments, a task that is becoming increasingly challenging due to the sheer volume of discoveries and a shortage of skilled technicians. The paper aims to automate this segmentation process by developing a method that can accurately divide 3D point cloud data of terracotta warrior fragments into their constituent parts (e.g., head, hand, foot). This automated segmentation would enable the creation of a database that assists archaeologists in efficiently matching excavated fragments, thereby accelerating the restoration workflow.

Methods

The proposed SRG-Net method consists of three main steps:

  1. Normal Estimation: The system first computes the normal vectors for each point in the 3D point cloud, using the xyz coordinates. This is a standard preprocessing step for many 3D point cloud analysis tasks.
  2. Seed-Region-Growing (SRG) Pre-segmentation: A novel unsupervised segmentation algorithm, SRG, is introduced. This method leverages the spatial continuity and similarity of normal vectors between neighboring points to coarsely segment the point cloud. It starts with randomly chosen "seed" points and iteratively grows regions by including neighboring points that meet specific criteria for normal and Euclidean distance similarity. This step aims to provide initial, rough segmentations.
  3. SRG-Net (CNN Segmentation and Refinement): A custom Convolutional Neural Network (CNN), named SRG-Net, is then employed. This network is inspired by DG-CNN (for dynamic graph learning) and FoldingNet (for auto-encoding). SRG-Net takes the pre-segmented point clouds as input and learns both local and global features to refine the segmentation. The network architecture includes a spatial transformer network, edge convolution layers for dynamic graph feature extraction, and graph convolution layers. A refinement step is also incorporated to further improve the segmentation quality by favoring spatially continuous clusters.

Datasets

  • Terracotta Warrior Dataset: The primary dataset consists of 500 intact terracotta warrior models collected using an Artec Eva scanner. For experiments, 400 models were used for training and 100 for validation. Each model was preprocessed to include xyz coordinates and vertical normals, and then uniformly sampled to 10,000 points. The dataset was also split into six parts to simulate fragmented pieces.
  • ShapeNet Dataset: This dataset was used to evaluate the robustness and generalizability of the SRG-Net method. It contains 16,881 objects across 16 categories, with models down-sampled to 2,048 points.

Key Insights and Ideas

  • Leveraging Normal Vectors for Unsupervised Segmentation: The paper demonstrates that normal vectors, in conjunction with spatial proximity, can be effectively used to guide unsupervised segmentation of 3D point clouds, even when color information is absent.
  • Synergy of SRG and CNN: The combination of a coarse, region-growing pre-segmentation (SRG) with a powerful deep learning model (SRG-Net) proves more effective than either method alone. The SRG step provides a good starting point, allowing the CNN to focus on refinement and learning complex features.
  • Dynamic Graph CNN for Point Clouds: The use of dynamic graph convolutional networks, which adapt their structure based on the input data, is crucial for capturing the intricate local geometric relationships within point clouds.
  • Refinement for Improved Cohesion: The inclusion of a refinement step significantly improves the segmentation by ensuring that points within a cluster are not only similar in features but also spatially contiguous, leading to more coherent and meaningful part segmentations.

Key Results

  • Terracotta Warrior Segmentation: SRG-Net achieved state-of-the-art results on the terracotta warrior dataset, outperforming methods like SRG-DGCNN, SRG-PointNet2, and SRG-PointNet in accuracy.
  • Accuracy and Latency: SRG-Net demonstrated a significant improvement in accuracy (e.g., over 5.5% increase compared to SRG-DGCNN) while consuming approximately 50% less latency.
  • ShapeNet Performance: On the ShapeNet dataset, SRG-Net achieved an overall accuracy of 82.63%, outperforming other methods and showing robustness across various object categories.
  • Ablation Study: The ablation study highlighted the importance of each component: graph convolution for topological features, edge convolution for local neighborhood relationships, and the refinement operation for adaptive clustering. The refinement operation alone increased accuracy by 15.6%.

Limitations and Future Work

  • Small Fragment Handling: The authors acknowledge that SRG-Net struggles with very small fragments, such as those from the hands of terracotta warriors, due to insufficient data for learning characteristics.
  • Future Work: The authors plan to address the issue of small fragments in future work. They also hope their method will be beneficial for archaeological research and the broader field of point cloud processing.

Summary updated

Abstract

The repairing work of terracotta warriors in Emperor Qinshihuang Mausoleum Site Museum is handcrafted by experts, and the increasing amounts of unearthed pieces of terracotta warriors make the archaeologists too challenging to conduct the restoration of terracotta warriors efficiently. We hope to segment the 3D point cloud data of the terracotta warriors automatically and store the fragment data in the database to assist the archaeologists in matching the actual fragments with the ones in the database, which could result in higher repairing efficiency of terracotta warriors. Moreover, the existing 3D neural network research is mainly focusing on supervised classification, clustering, unsupervised representation, and reconstruction. There are few pieces of researches concentrating on unsupervised point cloud part segmentation. In this paper, we present SRG-Net for 3D point clouds of terracotta warriors to address these problems. Firstly, we adopt a customized seed-region-growing algorithm to segment the point cloud coarsely. Then we present a supervised segmentation and unsupervised reconstruction networks to learn the characteristics of 3D point clouds. Finally, we combine the SRG algorithm with our improved CNN (convolution neural network) using a refinement method. This pipeline is called SRG-Net, which aims at conducting segmentation tasks on the terracotta warriors. Our proposed SRG-Net is evaluated on the terracotta warrior data and ShapeNet dataset by measuring the accuracy and the latency. The experimental results show that our SRG-Net outperforms the state-of-the-art methods. Our code is available at https://github.com/hyoau/SRG-Net.
BibTeX
@inproceedings{hu2021unsupervised,
  author = {Yao Hu and Guohua Geng and Kang Li and Wei Zhou and Xingxing Hao and Xin Cao},
  title = {{Unsupervised Segmentation for Terracotta Warrior with Seed-Region-Growing CNN(SRG-Net)}},
  booktitle = {CSAE},
  year = {2021},
  url = {https://openreview.net/forum?id=FnYl5sZH6x},
}
Figure