Introduction
Finding dense correspondence between 3D shapes—mapping every point on a source shape to its semantic equivalent on a target shape—is a foundational problem in 3D computer vision, crucial for animation, motion transfer, and alignment.
Historically, two main approaches have dominated, each with significant flaws:
- Spectral-based methods: These rely on the Laplace-Beltrami Operator (LBO) to align global geometric structures. While accurate, they require clean mesh connectivity (faces/edges) and heavy pre-processing, making them unsuitable for raw sensor data or real-time use.
- Spatial-based methods: These operate on raw point clouds using encoder-decoder networks. However, decoders that regress 3D coordinates often overfit to training distributions, leading to poor generalization on unseen shapes.
DPC (Deep Point Correspondence) introduces a real-time, unsupervised method that replaces traditional coordinate-regressing decoders with novel "cross-construction" and "self-construction" modules based on latent similarity.
Methods
The authors propose a completely unsupervised, decoder-free architecture. Instead of using a neural network to guess the 3D coordinates of the target shape, DPC uses the actual input coordinates and learns a highly discriminative feature space to match them.
The pipeline consists of three main components:
- Feature Extraction: A variant of DGCNN extracts high-dimensional features for each point. A static local neighborhood is used to prevent the network from confusing symmetric body parts (like left and right arms).
- Cross-Construction: To map a source point to a target, the model calculates the cosine similarity between the source point and target candidates in the learned latent space. It uses these weights to create a weighted average of the target points' actual 3D coordinates. The network is trained by minimizing the Chamfer Distance between this "constructed" shape and the actual target shape.
- Self-Construction: To ensure the learned features are spatially smooth, the model reconstructs each point using its latent neighbors within the same shape.
Key Insights and Ideas
- Decoders as a Bottleneck: By dropping the decoder and using an attention-like weighted sum of the existing input points (cross-construction), the network is freed to focus entirely on learning good matching features rather than memorizing geometry.
- Neural Proxy for Spectral Methods: The self-construction module acts similarly to the Laplace-Beltrami Operator used in spectral methods. It enforces smoothness in the feature space entirely in the learned latent space without needing actual mesh connectivity.
- Extreme Data Efficiency: Because the model isn't learning to reconstruct 3D geometry from scratch, it requires vastly less data. DPC can train on just 1% of the SURREAL dataset and still outperform competing models trained on the entire dataset.
Datasets and Evaluation
The authors evaluated their method on both human and animal datasets to test cross-dataset generalization:
- Human Shapes: Trained on SURREAL (synthetic parametric human shapes) and tested on SHREC'19 (real human scans).
- Animal Shapes: Trained on SMAL (synthetic parametric animals) and tested on TOSCA (diverse animal objects in various poses).
Key Results
- Accuracy: DPC significantly outperforms previous state-of-the-art point-based methods. On the SURREAL $\rightarrow$ SHREC'19 benchmark, DPC achieved 17.7% accuracy at a strict 1% error tolerance, compared to 6.0% for previous methods.
- Speed: DPC requires zero pre-processing and runs at 26.3 milliseconds per shape pair, making it roughly 100x faster than spectral methods.
- Generalization: DPC successfully trained on synthetic animals and accurately mapped dense correspondences onto entirely different, diverse animal meshes (TOSCA), achieving 33.2% accuracy at a 1% error tolerance.
Limitations and Future Work
The authors note that DPC struggles with extreme topological ambiguities, such as "self-contact" (e.g., hands touching hips) or highly unusual poses. Future work suggests that augmenting training data with these specific challenging scenarios could help the model learn to disambiguate these edge cases.
Summary updated