Abstract
We present ShapeFlow, a flow-based model for learning a deformation space for entire classes of 3D shapes with large intra-class variations. ShapeFlow allows learning a multi-template deformation space that is agnostic to shape topology, yet preserves fine geometric details. Different from a generative space where a latent vector is directly decoded into a shape, a deformation space decodes a vector into a continuous flow that can advect a source shape towards a target. Such a space naturally allows the disentanglement of geometric style (coming from the source) and structural pose (conforming to the target). We parametrize the deformation between geometries as a learned continuous flow field via a neural network and show that such deformations can be guaranteed to have desirable properties, such as be bijectivity, freedom from self-intersections, or volume preservation. We illustrate the effectiveness of this learned deformation space for various downstream applications, including shape generation via deformation, geometric style transfer, unsupervised learning of a consistent parameterization for entire classes of shapes, and shape interpolation.
Introduction
Traditional 3D representations in computer vision often face a trade-off between geometric detail and topological flexibility. Template-based models are excellent at preserving fine details but require consistent mesh connectivity across all instances. Conversely, generative models using voxels or implicit functions can represent arbitrary topologies but often result in "blobby" reconstructions that lose sharp CAD-like features. ShapeFlow addresses this gap by learning a calligraphic deformation space rather than a generative one, focusing on the continuous transformations between shapes to maintain high-fidelity geometric details.
Methodology
The core of ShapeFlow is treating deformation as an advection process. A 3D shape is morphed from a source to a target configuration by following a learned continuous vector field, effectively "blowing" the geometry into its new form.
Neural Flow Fields and Hub-and-Spoke Architecture
Deformations are parameterized as a continuous flow field $f_\theta(x, t)$ learned by a neural network. To handle the complexity of diverse shape categories (e.g., "all chairs"), the model utilizes a "hub-and-spoke" architecture. Every shape is mapped to a central "hub" or canonical latent code, avoiding the computational impossibility of learning pairwise transformations between every shape in a dataset.
Physical Constraints and Embedding
The framework incorporates physical constraints directly into the architecture to ensure realistic results:
- Invertibility: Ensures that deforming from Shape A to B and back to A returns to the original state.
- Volume Conservation: For organic models like humans, the flow is parameterized as the curl of a neural network's output, ensuring the volume remains constant during deformation.
- Auto-decoder Approach: Instead of a separate encoder, latent codes are optimized directly for each shape during training, allowing for precise embedding in the deformation space.
Key Insights and Applications
ShapeFlow's deformation-centric approach enables several unique capabilities that traditional generative models struggle with:
- Disentangling Style and Structure: "Style" is inherited from the source mesh (topology, sharp edges), while "structure" is dictated by the target (overall pose or proportions).
- Identity Preservation: Unlike traditional generative decoders that often blur inputs, deforming a shape to the hub and back preserves its original identity almost perfectly.
- Unsupervised Correspondence: By routing shapes through a common hub, the model naturally learns semantic correspondences across a class without manual labels.
Evaluation and Results
The authors evaluated ShapeFlow on ShapeNet (rigid objects) and SMPL (human bodies), demonstrating its versatility across different domains.
- Surface Reconstruction: ShapeFlow matches or exceeds state-of-the-art methods like OccFlow in reconstructing shapes from sparse point clouds, specifically excelling at preserving sharp corners and flat surfaces.
- Semantic Matching: On the ShapeNet chair dataset, ShapeFlow achieved a Semantic Matching Score (SMS) of 0.816, significantly higher than standard nearest-neighbor matching (0.779).
- Physical Plausibility: The volume-preservation constraint effectively prevents the "shrinking" effect common in linear interpolation, maintaining a near-constant volume ratio of 1.0.
Conclusion
ShapeFlow provides a robust framework for 3D modeling that respects both topological diversity and geometric precision. While it faces challenges regarding computational cost and occasional semantic misalignment in unsupervised settings, it represents a significant step toward high-fidelity 3D shape manipulation and reconstruction.
Summary updated