DeformSyncNet: Deformation transfer via synchronized shape deformation spaces

Minhyuk Sung, Zhenyu Jiang, Panos Achlioptas, Niloy J. Mitra, Leonidas J. Guibas

DeformSyncNet introduces a novel deep learning framework for synchronizing 3D shape deformations across object categories. Unlike traditional methods that rely on dense point-to-point correspondences, this approach learns a shared "canonical" latent space where deformations are represented as vectors. By decoding these vectors into shape-specific action dictionaries, the system enables consistent and plausible deformation transfer across diverse geometries, such as matching a swivel chair's movement to a four-legged stool.

The Challenge of 3D Deformation

Editing 3D geometry typically faces two major hurdles: the correspondence problem and structural plausibility. Dense correspondences—knowing exactly which point on one model matches another—are computationally expensive and often impossible for heterogeneous collections. Furthermore, arbitrary edits can easily "break" a model, causing parts to detach or destroying symmetry. DeformSyncNet addresses these by learning how a category of objects moves and changes, allowing for structure-preserving edits without pre-computed correspondences.

Methods: Synchronized Shape Deformation Space

The core of DeformSyncNet is its synchronized latent space. The architecture employs two primary components:

  1. The Encoder ($E$): Maps a 3D point cloud into a $k$-dimensional latent code. This space is treated as an affine space where a deformation is a vector ($v = E(y) - E(x)$).
  2. The Dictionary Predictor ($F$): Instead of reconstructing a shape, this network predicts a deformation dictionary ($A_x$) tailored to a specific source shape. Each column in this matrix represents a "basis" deformation, such as lengthening legs or widening a seat.
The DeformSyncNet architecture: An encoder $E$ maps shapes to a latent space, while a predictor $F$ generates a shape-specific deformation dictionary.

The "Sync" mechanism allows a latent deformation vector from a source pair to be applied to a target shape using the target's unique dictionary. A sparsity regularization loss ($L_{2,1}$) ensures the network finds meaningful, localized dictionary elements rather than global distortions.

Key Insights

  • Deformation as an Action: The model focuses on reconstructing the edit rather than the shape itself, effectively learning the "physics" of how specific geometries can change.
  • Path Invariance: The affine nature of the latent space ensures mathematical consistency (the "parallelogram law"), keeping transfers stable across multiple edits.
  • Projection to Plausibility: The system can "snap" user inputs to the nearest plausible version of a shape recognized by the network, automatically maintaining symmetry and connectivity.

Evaluation and Results

The framework was evaluated on ShapeNet categories (Airplane, Car, Chair, Sofa, Table) and parametric models. DeformSyncNet outperformed state-of-the-art methods like 3DN and Neural Cages in fitting accuracy (measured by Chamfer Distance).

Deformation transfer and projection: Edits on a source shape are projected to a plausible subspace and transferred to diverse target models.

In user studies, participants preferred DeformSyncNet's transfers 54.7% of the time. Quantitatively, the model demonstrated significantly higher consistency in the "parallelogram test," with error rates 3-10x lower than competitors, proving the robustness of its synchronized latent space.

Conclusion

DeformSyncNet successfully bridges the gap between diverse 3D models within a category. While currently limited to global point clouds and single-category training, future work could incorporate part-level features and hierarchical constraints to handle even finer details and cross-domain transfers.

Summary updated

BibTeX
@article{sung2020deformsyncnet,
  author = {Minhyuk Sung and Zhenyu Jiang and Panos Achlioptas and Niloy J. Mitra and Leonidas J. Guibas},
  title = {{DeformSyncNet: Deformation transfer via synchronized shape deformation spaces}},
  journal = {ACM Trans. Graphics},
  year = {2020},
  doi = {10.1145/3414685.3417783},
}
Figure