ShapeFlow: Learnable Deformation Flows Among 3D Shapes

Chiyu Max Jiang, Jingwei Huang, Andrea Tagliasacchi, Leonidas J. Guibas

In the world of 3D computer vision, shapes are typically represented in one of two ways: as rigid "morphable models" (where variations are limited to sliding predefined parameters, like nose length on a digital face) or as "generative models" that attempt to dream up a shape from scratch. The former is often too restrictive, while the latter frequently loses the crisp, fine details that make a 3D model look realistic and professional.

ShapeFlow, presented at NeurIPS 2020, offers a compelling third path. Instead of learning how to build a shape from the ground up, it learns how to deform one shape into another using continuous, fluid-like motions.

Executive Summary

ShapeFlow is a flow-based deformation model that learns to transform 3D shapes by advecting them through a learned vector field. By focusing on deformation rather than direct generation, it preserves fine geometric details and naturally disentangles an object's "style" from its "structure." Furthermore, it guarantees physical properties like volume preservation and the prevention of self-intersections.

Overview of the ShapeFlow framework, showing how input observations are embedded into a learned deformation space to enable applications like shape correspondence, reconstruction, and geometric style transfer.

The Challenge of 3D Representation

Traditional 3D generative models—such as those relying on voxels or implicit functions—often struggle with "over-smoothing." While they might capture the general silhouette of a chair, sharp edges and intricate details (like thin wooden slats) are frequently lost during the decoding process.

Conversely, template-based models (like the SMPL model for human bodies) excel at preserving detail but require every shape to share the exact same "topology." This means a human body template cannot be easily repurposed to model a chair, nor can a standard four-legged chair template easily morph into a swivel chair.

The core challenge ShapeFlow addresses is: How can we create a shared representation for an entire class of objects that is flexible enough to handle varying topologies, yet precise enough to retain the fine details of the original CAD models?

The ShapeFlow Methodology

ShapeFlow tackles this by treating shape transformation as a continuous "flow" problem.

  1. Continuous Flow Fields: Instead of predicting where each vertex of a mesh should jump in a single step, ShapeFlow trains a neural network to define a 3D vector field. Points on a source shape are "pushed" along this field over a simulated time interval ($t=0$ to $t=1$) until they reach the target configuration.
  2. The Hub-and-Spoke Model: Learning direct transformations between every possible pair of shapes would result in an unmanageable $N^2$ complexity. To solve this, ShapeFlow uses a "hub" approach. Every shape in a category is learned as a deformation to and from a single, central "canonical" shape (the hub).
  3. Mathematical Guarantees:
    • Invertibility: Because the flow is continuous, the transformation can be run backward to return exactly to the starting point.
  4. No Self-Intersections: Under ideal conditions, the continuous flow prevents the mesh from "folding into itself," a common artifact in 3D modeling.
  5. Volume Preservation: By utilizing a "divergence-free" vector field (calculated as the curl of the network's output), the model can force the object's volume to remain constant during the morph—a crucial feature for realistic animations.

Key Insights and Applications

  • Style vs. Structure Disentanglement: In ShapeFlow, "style" (fine details, mesh connectivity, specific parts) is derived from the source shape. The "structure" (height, width, general pose) is dictated by the latent code controlling the flow. This makes geometric style transfer as simple as deforming a highly detailed, ornate chair into the broader proportions of a plain one.
  • Topology Agnostic: Because the flow field exists in the 3D space around the shape, the source format is largely irrelevant. Whether the input is a mesh, a point cloud, or a collection of voxels, the flow simply moves the underlying space itself.
  • Unsupervised Correspondence: By deforming all objects in a category to a central "hub," the model implicitly learns which part of one object corresponds to another (e.g., mapping the leg of chair A to the leg of chair B) without requiring manual semantic labels.

Results

The model was evaluated on standard benchmarks including ShapeNet (chairs, airplanes, cars) and the SMPL dataset for human body shapes.

  • Detail Preservation: Qualitatively, ShapeFlow produces significantly sharper results than generative baselines like Occupancy Networks. Because it deforms an existing high-quality CAD model, the output retains a crisp, "CAD-like" appearance.
  • Quantitative Performance: On ShapeNet reconstruction tasks, ShapeFlow achieved a highly competitive Chamfer-L1 distance of 0.1378 (mean across categories), offering superior visual fidelity.
  • Semantic Matching: In unsupervised correspondence tests, ShapeFlow achieved a Semantic Matching Score of 0.816, significantly outperforming the standard ShapeNet baseline of 0.779. This confirms that the learned "hub" space is semantically meaningful.

Limitations and Future Work

While powerful, ShapeFlow is currently "semantically blind." If two shapes are drastically different, the model might accidentally morph a chair leg into a backrest simply because it represents the mathematically shortest path in the flow field.

The authors suggest that future iterations could benefit from part-based logic—segmenting shapes and learning specific vector fields for individual components (e.g., a dedicated "leg flow" and "seat flow"). Additionally, there is exciting potential to apply this framework to inverse problems in engineering physics, such as inferring fluid dynamics by observing how a shape deforms within a medium.

Summary updated

Abstract

We present ShapeFlow, a flow-based model for learning a deformation space for entire classes of 3D shapes with large intra-class variations. ShapeFlow allows learning a multi-template deformation space that is agnostic to shape topology, yet preserves fine geometric details. Different from a generative space where a latent vector is directly decoded into a shape, a deformation space decodes a vector into a continuous flow that can advect a source shape towards a target. Such a space naturally allows the disentanglement of geometric style (coming from the source) and structural pose (conforming to the target). We parametrize the deformation between geometries as a learned continuous flow field via a neural network and show that such deformations can be guaranteed to have desirable properties, such as bijectivity, freedom from self-intersections, or volume preservation. We illustrate the effectiveness of this learned deformation space for various downstream applications, including shape generation via deformation, geometric style transfer, unsupervised learning of a consistent parameterization for entire classes of shapes, and shape interpolation.
BibTeX
@inproceedings{jiang2020shapeflow,
  author = {Chiyu Max Jiang and Jingwei Huang and Andrea Tagliasacchi and Leonidas J. Guibas},
  title = {{ShapeFlow: Learnable Deformation Flows Among 3D Shapes}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
  url = {https://openreview.net/forum?id=Jpl-18iaKg},
}
Figure