Isometric Shape Representation by Integrating Shape Function Maps and Deep Learning

Zijian Wang, Zhenzhong Kuang, Zhiqiang Guo, Suguo Zhu, Min TanIEEE Access 2019 · DOI

Abstract

This paper presents a novel hybrid framework for 3D shape recognition that bridges the gap between traditional geometric descriptors and modern deep learning. By utilizing "Shape Function Maps" (SFMs) to encode both rigid Euclidean and non-rigid intrinsic geometry into 2D representations, the proposed method enables shallow convolutional neural networks (CNNs) to achieve state-of-the-art performance. The approach, particularly the Deep Stacking SFM (DS-SFM) variant, demonstrates superior accuracy and robustness across diverse datasets, including human poses, CAD models, and biological structures, while requiring significantly less training data than end-to-end deep learning models.

The Challenge of Isometric Deformation

A fundamental problem in 3D computer vision is the representation of isometric shapes. An isometric deformation occurs when an object changes its pose in a way that preserves distances along its surface—for instance, a human bending a limb or a hand forming different gestures. While the underlying object remains identical, its raw 3D coordinates change dramatically, posing a significant challenge for standard recognition algorithms.

Examples of isometric deformations in 3D shapes, including humans, hands, and synthetic objects.

Existing approaches typically fall into two categories: hand-crafted descriptors and pure deep learning. Hand-crafted methods, often based on mathematical principles like heat diffusion, are robust to bending but may fail to capture subtle discriminative features. Conversely, pure deep learning models like PointNet can learn features directly from data but often require massive datasets to achieve pose invariance and can be sensitive to noise or low resolution.

Methodology: Integrating Geometry with Deep Learning

The researchers propose a "mixture modeling" approach that pre-organizes geometric data into structured 2D maps before processing them with a CNN. This pipeline consists of three primary components:

1. Shape Function Maps (SFM)

Instead of feeding raw point clouds into a network, the authors generate 2D maps representing pairwise vertex relationships:

  • Non-rigid SFM (NSFM): Encodes intrinsic coordinates based on the Laplace-Beltrami operator, capturing features invariant to bending.
  • Rigid SFM (RSFM): Uses standard Euclidean distances and angles to capture the rigid, global structure of the object.
  • Joint SFM (JSFM): A composite map designed to integrate both rigid and non-rigid information.

2. Shallow CNN Architecture

Because the SFMs are highly structured and informative, the framework does not require a deep, complex network. A relatively simple CNN with five convolutional layers is sufficient to extract highly discriminative features from the maps, leading to faster training and lower data requirements.

3. Deep Stacking (DS-SFM)

The "Deep Stacking" variant is the framework's most advanced iteration. It involves training separate networks for rigid and non-rigid maps and then stacking their features in a final fusion layer. This allows the model to dynamically weigh the importance of pose-invariant features versus rigid structural features based on the specific characteristics of the dataset.

Experimental Results and Discussion

The framework was evaluated across a broad spectrum of benchmarks, demonstrating its versatility and effectiveness.

  • High Classification Accuracy: On the SHREC15 dataset, the DS-SFM model achieved a classification accuracy of 98.7%, significantly outperforming established methods such as DeepGM (93.0%) and 3DMMF (95.4%).
  • Robust Retrieval: The method showed exceptional performance in shape retrieval tasks, particularly on the McGill and SHREC11 datasets, maintaining high Nearest Neighbor (NN) and First Tier (FT) scores.
  • Cross-Domain Versatility: The model's effectiveness extended to biological data. On the SHREC18 protein dataset, it achieved an NN score of 87.6%, surpassing PointNet's 82.6%.
  • Noise Resistance: Due to the use of the Heat Kernel in calculating intrinsic coordinates, the method is naturally resistant to surface noise and vertex displacement.

Conclusion and Future Directions

By combining the strengths of geometric modeling and deep learning, this research provides a robust solution for isometric shape representation. The use of SFMs effectively "teaches" the network about pose invariance, reducing the burden on the learning process.

While the current framework is highly effective, future work could address the computational cost of calculating Laplace-Beltrami eigenfunctions for very high-resolution meshes. Additionally, exploring more complex fusion strategies and extending the method to dynamic 3D video sequences or real-time gesture recognition remains a promising avenue for further development.

Summary updated

BibTeX
@article{wang2019isometric,
  author = {Zijian Wang and Zhenzhong Kuang and Zhiqiang Guo and Suguo Zhu and Min Tan},
  title = {{Isometric Shape Representation by Integrating Shape Function Maps and Deep Learning}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2950279},
}
Figure