TL;DR
This paper introduces the first triangulation-agnostic generative framework for 3D triangle meshes by combining flow matching with a novel noise distribution based on Matérn processes. By ensuring the initial noise is independent of the mesh's specific discretization, the model can be trained on low-resolution data and successfully generate highly realistic deformations on high-resolution meshes with over a million triangles at test time.
What Problem Does This Paper Address?
Generative modeling for 3D meshes has historically lagged behind modalities like images or voxels. A major roadblock is triangulation sensitivity. Unlike regular pixel grids, meshes are irregular graphs representing arbitrary discretizations of a 3D surface.
Traditional denoising generative models (like diffusion or flow matching) rely on independent and identically distributed (iid) Gaussian noise sampled at each vertex. However, if a mesh's triangulation or resolution changes at test time, this naive noise distribution changes completely, causing the trained denoiser to fail. To make mesh generation practical for real-world applications—where datasets contain heterogeneous triangulations and inference is often desired on high-resolution geometry—we need a framework where both the network architecture and the underlying noise distribution are completely agnostic to how the mesh is discretized.
Methods
The authors tackle this challenge by formalizing what it means for a noise distribution to be triangulation-agnostic using spectral shape analysis, leading to a complete generative pipeline:
- Spectral Definition of Agnosticism: The paper outlines three essential spectral properties for a noise distribution to be triangulation-agnostic: per-frequency independence, mesh-invariant frequency statistics, and bounded high-frequency content. Naive Gaussian noise fails because it injects unbounded high-frequency energy as mesh density increases.
- Matérn Noise: The authors demonstrate that a discretization of a Gaussian random field called a Matérn process perfectly satisfies these properties. Instead of injecting independent noise at each vertex, Matérn noise acts like a low-pass filter over the mesh spectrum. Sampling is performed efficiently by solving a screened Poisson equation using standard sparse Cholesky solvers.
- Scale Invariance: They introduce a normalization scheme based on the Frobenius norm of the discrete Laplace-Beltrami operator to ensure the noise remains statistically consistent across different shapes and global scales.
- Flow Matching Pipeline: The complete architecture utilizes the Flow Matching (FM) paradigm to define a vector field that interpolates between the Matérn noise ($\mathbf{f}_0$) and the target data ($\mathbf{f}_1$). The denoiser backbone is PoissonNet, a state-of-the-art architecture that operates in the gradient domain to ensure triangulation-agnostic learning.
Datasets
The method was evaluated on three distinct types of data:
- MOYO Dataset: Contains SMPL human meshes in complex yoga poses (18k faces), used to evaluate single-source deformation and generalization to entirely unseen human shapes.
- Elastic Deformations Dataset: Generated using physical simulations (Incremental Potential Contact) of dropping 3D objects (Octopus, Elephant, Fish, Stanford Bunny) until they reach equilibrium. Training was performed on low-resolution versions (9k–10k faces) to simulate real-world data constraints.
- Synthetic GMM Dataset: A synthetic dataset consisting of random Gaussian Mixture Model signals on a Stanford Bunny mesh, used to evaluate the statistical accuracy of the generated fields.
Key Insights and Ideas
- Architecture Isn't Enough: Simply using a triangulation-agnostic neural network (like DiffusionNet or PoissonNet) is insufficient if the input noise distribution itself changes with the mesh resolution. The noise model must be fundamentally tied to the continuous geometry, not the discrete graph.
- The Screened Poisson Equation as a Filter: Solving a screened Poisson equation turns out to be an incredibly elegant way to sample smooth, mesh-independent noise. It naturally dampens high-frequency variations that would otherwise disrupt the denoiser on dense meshes.
- Resolution Independence: Because the noise spectrum decays predictably at high frequencies, higher-resolution meshes do not inject chaotic high-frequency artifacts. This enables zero-shot upsampling, allowing a model trained on a 9k-face mesh to generate clean results on a 100k-face mesh.
Key Results
- High-Resolution Generation: The model successfully generates realistic deformations on meshes exceeding 1 million triangles, even when trained exclusively on low-resolution data.
- Outperforming Baselines: In human pose generation, the proposed method significantly outperforms non-agnostic mesh baselines like Manifold Diffusion Fields (MDF) and DoubleDiffusion, achieving much lower Mean Squared Error (MSE) and Minimum Matching Distance (MMD).
| Method | MSE $\cdot 10^4 \downarrow$ | MMD $\downarrow$ | COV $\uparrow$ |
|---|---|---|---|
| MDF | 1411 | 0.5121 | 0.051 |
| DoubleDiffusion | 155 | 0.1009 | 0.387 |
| Ours (full) | 0.27 | 0.0693 | 0.487 |
- Robustness to Corruption: The framework remains highly effective even when the source mesh undergoes topological corruption (e.g., cutting holes into the surface) or severe triangulation deterioration (e.g., random edge flips).
Limitations and Future Work
- Backbone Bottleneck: The denoiser (PoissonNet) cannot capture extreme high-frequency details, preventing the generation of sharp features like crisp RGB textures directly on meshes. It is also currently limited to single connected components and 2-manifolds.
- Physical Inaccuracies: While the model emulates elastic deformations convincingly, it does not enforce strict physical constraints, occasionally leading to minor self-penetrations or ground-plane collisions.
- Future Directions: Promising avenues include extending the framework to generate temporal animations, applying it to volumetric data for medical imaging, and exploring pixel-agnostic image generation.
Summary updated