In the early days of deep learning for 3D geometry, the most intuitive way to generate a mesh was to treat it like digital clay. You start with a simple, well-behaved primitive—usually a sphere or an ellipsoid—and train a neural network to "push" and "pull" its vertices until it matches a target shape.
This approach, pioneered by works like Pixel2Mesh by Nanyang Wang et al., is mathematically elegant because it preserves the underlying connectivity of the mesh. However, it suffers from a fundamental geometric limitation: it is topologically rigid. If you start with a sphere, you can only ever end with something that is topologically equivalent to a sphere.
The Homeomorphism Trap
The core of the problem lies in the concept of genus. In topology, a sphere has a genus of zero (no holes), while a donut (torus) has a genus of one. As Xian-Feng Han and colleagues note in their comprehensive survey, a continuous deformation—the kind performed by most Graph Convolutional Networks (GCNs)—cannot "tear" the mesh to create a hole or "weld" parts together to change the genus.
When a model like Pixel2Mesh tries to reconstruct a chair with a gap between the backrest and the seat, it cannot actually create that gap. Instead, it must stretch the "clay" extremely thin to approximate the void. This leads to what Éloi Mehr describes in DiscoNet as mesh tangling and poor vertex correspondence. The network is forced to fight against the very structure it was given.
Bridging the Gap with Skeletons and Voxels
To move beyond the "sphere-to-object" paradigm, researchers have introduced intermediate representations that act as topological blueprints.
One powerful strategy is the skeleton-bridged approach. Jiapeng Tang and Xiaoguang Han proposed a method in A Skeleton-Bridged Deep Learning Approach that first predicts a 3D "meso-skeleton." Because a skeleton is a graph of points and lines, it can easily represent complex topologies, including holes and disconnected parts. Once the topology is captured by the skeleton, the mesh is built around it, ensuring the final surface respects the object's true structure.
Similarly, Mesh R-CNN bypasses the fixed-template problem by starting with voxels. Unlike a mesh, a voxel grid doesn't care about connectivity; it simply marks which parts of space are "full." By converting a predicted voxel grid into a coarse mesh and then refining it, the model can represent objects with arbitrary numbers of holes and disjoint components.
From Deformation to Generation
A more radical shift involves abandoning the idea of "deforming" altogether in favor of "generating" the mesh graph from scratch.
Scan2Mesh, developed by Angela Dai and Matthias Nießner, uses a generative framework to directly predict vertices and their connections. By treating the mesh as a graph to be built rather than a surface to be stretched, the model can produce "artist-like" meshes that are compact and topologically accurate.
In a similar vein, REIN by Rangel Daroya uses a recurrent neural network to infer edges sequentially. This allows the mesh to grow organically, vertex by vertex, freeing it from the constraints of a predefined base shape.
The Rise of Implicit Representations
Perhaps the most significant trend in overcoming topological constraints is the move toward Implicit Neural Representations (INRs). Instead of explicitly defining vertices and faces, methods like Differentiable Volumetric Rendering (DVR) by Michael Niemeyer and Lars M. Mescheder represent shape as a continuous function (like an occupancy field).
In an implicit representation, the "mesh" is just an isosurface extracted from a field. If the field changes such that two parts of the shape meet or a hole opens up, the topology updates automatically and fluidly. This allows for the reconstruction of high-resolution, complex geometries—like the interior of a mouth in ScanTalk or the fluid dynamics of a splashing liquid—that would be impossible to capture with a fixed-topology template.
By moving from rigid deformation to flexible generation and implicit fields, the field is transitioning from simply "wrapping" objects to truly understanding and synthesizing their fundamental structure.