Most successful deep learning architectures, from Convolutional Neural Networks (CNNs) to Transformers, were designed for data that lives on a regular grid. Images are rectangular arrays of pixels, and text is a linear sequence of tokens. In these domains, the neighborhood of any point is fixed and predictable: a pixel in the middle of an image always has exactly eight neighbors.
However, 3D meshes—the industry standard for representing geometry—are fundamentally non-Euclidean. They consist of vertices, edges, and faces arranged in irregular topologies. As noted by Gao et al., this irregularity prevents models from naturally learning spatial correlations because there is no consistent grid to define a sliding filter. A single vertex in a mesh might be shared by three faces or thirteen, making traditional convolution operations mathematically undefined.
The Breakdown of Translation Invariance
In image processing, we rely on translation invariance: a filter that detects an edge in the top-left corner of an image should work exactly the same way in the bottom-right. On a curved 3D surface, this concept vanishes. As Masci et al. explain, manifolds lack a global coordinate system, meaning there is no "up" or "right" that applies consistently across the entire shape.
Researchers have proposed three primary ways to bridge this gap: spectral methods, spatial kernels, and domain mapping.
Spectral and Spatial Reformulations
One approach is to move the problem into the spectral domain. Bruna et al. pioneered the use of the graph Laplacian to define convolutions through the lens of spectral properties. By analyzing the "vibrations" of the graph, they created deep architectures that maintain parameter efficiency without needing a regular grid.
Alternatively, spatial methods attempt to redefine the "patch" locally. Masci et al. introduced Geodesic Convolutional Neural Networks (GCNNs), which use polar coordinates on the surface to create a local patch that is invariant to how the shape bends or deforms. Similarly, Feng et al. proposed MeshNet, which shifts the focus from vertices to polygon faces as the fundamental unit, splitting features into spatial and structural components to handle inconsistent connectivity.
Mapping to Regular Domains
A different strategy involves forcing irregular 3D data back into a regular 2D format. This allows researchers to leverage powerful, pre-trained 2D models. For instance, Taha et al. created "Fused Geometrically Augmented Images" by projecting 3D facial descriptors onto 2D texture maps. Fan et al. used a "Geometric Map" to project irregular facial meshes onto a symmetric square grid, enabling the use of standard GAN architectures for high-fidelity shape manipulation.
Generative Modeling of Surfaces
Generating new meshes is even more difficult than classifying existing ones. The discrete, combinatorial nature of mesh connectivity—knowing which vertex connects to which—is hard for neural networks to predict.
Fisher et al. addressed this with AtlasNet, which treats a 3D shape like a "papier-mâché" collection of parametric patches. Instead of predicting a global grid, the network learns to deform a set of 2D squares into 3D space. To handle more complex, structured objects, Gao et al. developed SDM-NET, which uses a two-level Variational Autoencoder to jointly encode the global structure of an object and the local geometry of its deformable parts.
More recently, the field has moved toward continuous representations. Zhang introduced Functional Diffusion, which treats data as infinite-dimensional functions rather than discrete points or pixels. This allows for the generation of complex deformations across irregular domains without being tied to a specific mesh resolution or vertex count.
Handling Real-World Irregularity
While synthetic CAD models are often clean, real-world data is messy. Boscaini et al. point out that meshes generated from photogrammetry or smartphones often contain noise, holes, and varying point densities that break models trained on perfect data.
To combat this, newer architectures focus on topology-agnosticism. Bône et al. developed a VAE that uses "current-splatting" and "exponentiation" layers to process meshes regardless of their connectivity or noise levels. Similarly, Peng et al. introduced MWFormer, which uses a Breadth-First Search (BFS) pooling strategy to group irregular faces into tokens, allowing Transformers to capture long-range dependencies in 3D geometry just as they do in text.