Abstract
Extracting skeletal structures from unordered point clouds is a challenging task in computer vision due to the lack of fixed topology. This paper introduces PSPU-SkelNet, a deep learning architecture that treats 2D shape point clouds as image-based data. By employing a pyramid of three U-Nets with varying receptive fields, the model effectively captures multi-scale geometric features. The proposed method achieved 3rd place in the Point SkelNetOn 2019 challenge, demonstrating a Chamfer Distance of 2.9105.
Introduction
The "skeleton" or medial axis of a shape provides a compact representation useful for character animation, shape matching, and structural analysis. While skeletonization is well-established for 2D pixel grids, it becomes significantly more complex when the input is a point cloud—an unordered set of $(x, y)$ coordinates typical of laser scanners and depth sensors. These inputs lack the inherent connectivity of images, making it difficult to predict thin, representative skeletal structures without losing geometric essence.
Methods
The core innovation of PSPU-SkelNet is the "deserialization" of point clouds into 2D images, bridging the gap between point cloud processing and traditional computer vision.
Image-Based Processing
Instead of using architectures designed for raw points (e.g., PointNet), the authors convert point clouds into 2D images. This transformation allows the model to leverage the mature capabilities of 2D Convolutional Neural Networks (CNNs). The authors found that this image-based approach outperformed native 1D point cloud convolutions, which often produced outliers and struggled with variable point-cloud lengths.
Pyramid U-Net Architecture
The system utilizes a pyramid of three U-Nets, each operating at a different stride (2, 4, and 8). This multi-scale approach, inspired by PSPNet, enables the network to simultaneously extract fine local details and global structural information.
Implementation Details
- Instance Normalization: Replaced batch normalization to better suit the image-to-image translation nature of the task.
- Loss Function: The model was trained using binary cross-entropy loss and optimized with the Adam optimizer.
- Data Augmentation: To address the limited size of the Point SkelNetOn 2019 dataset (1,219 training shapes), the authors applied extensive augmentation (translation, rotation, scaling, flipping), increasing the training volume by 26x.
Results
The model was evaluated on the Point SkelNetOn 2019 challenge dataset.
Quantitative Performance
TeamPH ranked 3rd overall in the challenge, achieving a Chamfer Distance (CD) of 2.9105 on the final test set of 266 shapes.
Qualitative Performance
Visual analysis confirms that the network effectively predicts skeletons for complex shapes, maintaining high fidelity to ground truth. However, performance slightly degrades for shapes with extremely low point densities.
Discussion and Lessons Learned
The authors explored several alternative strategies that proved less effective:
- Generative Adversarial Networks (GANs): Did not improve Chamfer Distance and introduced unwanted outliers.
- MAE Loss: Resulted in thicker, less accurate skeletons compared to binary cross-entropy.
- Dropout: Found to lower overall prediction scores.
Limitations
With 57 million parameters, the network is computationally heavy. Its performance is also sensitive to point density, suggesting a need for greater robustness to sparse data. Future work will focus on model pruning and compression to make the architecture more practical for standard hardware.
Conclusion
PSPU-SkelNet demonstrates that treating 2D point clouds as images and applying a multi-scale U-Net architecture is a highly effective strategy for skeleton extraction. By reconciling local and global information through a pyramid structure, the model achieves competitive results on benchmark datasets.
Summary updated