Continuous Latent Manifolds for 3D Shape Morphing

This direction is split into other directions: latent manifold interpolation. Its last version stays readable and citable here.

Version 1 of 1163 papers189 conceptsfrozen 2026-10-04

In traditional computer graphics, "morphing" one 3D object into another is a geometric nightmare. If you simply average the coordinates of a chair and a table, you don't get a "chable"—you get a cloud of floating points or a self-intersecting mess of polygons. This happens because the raw representation of 3D data (voxels, meshes, or point clouds) is not "aware" of the underlying structure of the objects it represents.

Recent research has shifted toward representing 3D geometry within continuous latent spaces. By encoding complex shapes into compact, high-dimensional vectors, researchers are creating manifolds where the distance between points corresponds to semantic similarity, and the paths between them represent physically plausible transformations.

The Problem with Discrete Geometry

The fundamental challenge in 3D generation is that geometric data is often unstructured or discrete. Point clouds are unordered sets; meshes have fixed topologies that break during deformation; and voxels are computationally expensive. When Maxim Tatarchenko and Alexey Dosovitskiy explored Multi-view 3D Models from Single Images, they demonstrated that a network could learn to "imagine" unseen views by encoding geometry into a continuous latent space. In this space, linear interpolation between two vectors doesn't just average numbers; it navigates a learned manifold of "chair-ness" or "car-ness."

This shift from raw coordinates to learned embeddings is what allows models to generalize. As Yuheng Liu and colleagues argue in their work on 3D Gaussian Splatting, raw 3D parameters are often "messy" and inconsistent. By mapping these parameters to a unified "submanifold field," they eliminate the visual jitter and artifacts that occur when trying to blend different geometric states.

Ensuring Physical Plausibility

A major hurdle in latent interpolation is ensuring that the intermediate states—the "in-between" shapes—are valid. If you interpolate between two human poses, the arms shouldn't pass through the torso.

Lin Gao addressed this in Variational Autoencoders for Deforming 3D Mesh Models by using a rotation-invariant feature space (RIMD). Because the latent space is organized around local deformations rather than global coordinates, the model can synthesize new shapes that maintain surface integrity and avoid self-intersections.

Similarly, Ruqi Huang and Leonidas Guibas introduced OperatorNet, which treats shape differences as mathematical operators. Instead of linear blending, they perform interpolation in the associated Lie algebra (using matrix logarithms and exponentials). This ensures that the transition between shapes is distortion-free, preserving the "volume" and "character" of the object throughout the morph.

Alignment as a Prerequisite for Morphing

For a latent space to be truly continuous, the objects within it must be "aligned" in the model's mind. If one chair is facing north and another is facing south, the midpoint in latent space might be a collapsed blob.

Keyang Zhou and Gerard Pons-Moll solved this with the Adjoint Rigid Transform (ART) Network, which automatically aligns 3D shapes into a canonical orientation before encoding them. This ensures that when you interpolate between a sedan and an SUV—as seen in Ivan Kostiuk's HyperColor—the wheels of the first map directly to the wheels of the second, preventing geometric "squeezing."

This concept of alignment is taken further by Savva Ignatyev in A3D. By forcing a diffusion model to optimize multiple objects within a shared latent space, the model learns to align semantic features (like limbs or joints) across different generated assets, enabling seamless "hybridization."

From Diffusion to Deterministic Flows

While Diffusion Models have become the standard for 3D generation, they are inherently stochastic—adding noise makes it difficult to find a "straight line" between two shapes. J. Ryan Shue and Gordon Wetzstein showed that Neural Field Diffusion can bridge this gap by operating on "triplanes," but the transitions can still be unpredictable.

The most recent frontier involves Flow-based models. Minghao Yin and Kai Han introduced WUKONG, which uses rectified flow models to treat morphing as a deterministic trajectory along an Ordinary Differential Equation (ODE). This ensures that every intermediate step between two objects remains exactly on the "manifold of realism," allowing for high-fidelity, textured 3D morphing that looks like a natural evolution rather than a cross-fade.

Beyond Visuals: Molecules and CAD

The power of continuous 3D latent spaces extends beyond graphics into functional design. In MolFLAE, Zitao Chen and Wei-Ying Ma map 3D molecules into an E(3)-equivariant latent space. Because the space is regularized, researchers can perform "vector math" on drugs—adding a "solubility" vector to a molecule's latent representation to smoothly transform its chemical properties while maintaining its 3D structure.

Similarly, SECAD-Net by Pu Li applies these concepts to CAD models. By mapping mechanical parts to a continuous neural space, engineers can blend different components into novel, hybrid geometries that remain structurally coherent and editable.

By treating 3D geometry not as a collection of points, but as a coordinate on a learned, continuous manifold, these researchers have turned the "chable" from a geometric impossibility into a simple walk through latent space.

Papers in this version

Showing 60 of 163 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_continuous_latent_manifolds_for_3d_shape_46d2e10e,
  title = {Continuous Latent Manifolds for 3D Shape Morphing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/continuous-latent-manifolds-for-3d-shape-morphing/rdr_46d2e10eb00340d69095c06185f734e0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_continuous_latent_manifolds_for_3d_shape_46d2e10e_v1,
  title = {Continuous Latent Manifolds for 3D Shape Morphing},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/continuous-latent-manifolds-for-3d-shape-morphing/rdr_46d2e10eb00340d69095c06185f734e0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}