Abstract: Micro-mesh (μ-mesh) is a new graphics primitive for compact representation of extreme geometry, consisting of a low-polygon base mesh enriched by per micro-vertex displacement. A new generation of GPUs supports this structure with hardware evolution on μ-mesh ray tracing, achieving real-time rendering in pixel level geometric details. In this article, we present a differentiable framework to convert standard meshes into this efficient format, offering a holistic scheme in contrast to the previous stage-based methods. In our construction context, a μ-mesh is defined where each base triangle is a parametric primitive, which is then reparameterized with Laplacian operators for efficient geometry optimization. Our framework offers numerous advantages for high-quality μ-mesh production: (i) end-to-end geometry optimization and displacement baking; (ii) enabling the differentiation of renderings with respect to μ-mesh for faithful reprojectability; (iii) high scalability for integrating useful features for μ-mesh production and rendering, such as minimizing shell volume, maintaining the isotropy of the base mesh, and visual-guided adaptive level of detail. Extensive experiments on μ-mesh construction for a large set of high-resolution meshes demonstrate the superior quality achieved by the proposed scheme.
The Micro-Mesh Challenge
As 3D scenes become increasingly complex, standard triangle meshes struggle with memory overhead and rendering performance. Enter the micro-mesh ($\mu$-mesh): a hybrid primitive consisting of a low-polygon "base mesh" and a dense grid of scalar displacements (micro-vertices) applied at render-time. This format allows for extreme geometric detail with a fraction of the memory footprint of traditional high-poly meshes.
Until now, creating these meshes involved a "staged" pipeline: first decimate the original model, then remesh it for uniformity, and finally "bake" the displacements. The problem is that each stage optimizes for a local goal (like geometric error) without considering the final rendering quality or the efficiency of the GPU's ray-tracing hardware. This often leads to "tangled" geometry, oversized bounding shells that slow down ray tracing, and a lack of flexibility in capturing fine details.
A Holistic Differentiable Framework
The authors propose a holistic, differentiable construction system. Instead of a sequence of fixed steps, they treat $\mu$-mesh construction as an optimization problem that can be solved using gradient descent. By formulating the subdivision and displacement process as a differentiable computation graph, gradients can flow from the final rendered image back to the base mesh positions and displacement values.
Key technical contributions include:
- Laplacian Reparameterization: To prevent the mesh from becoming "tangled" or distorted during optimization, the framework uses Laplacian operators. This ensures that vertex movements are smooth and preserve the local neighborhood structure.
- Novel Objective Functions:
- Reprojectability: A loss based on rendering consistency across multiple views, ensuring the $\mu$-mesh looks identical to the high-poly original.
- Shell Volume Minimization: A specific regularizer that shrinks the "bounding shell" of the mesh, which directly improves GPU ray-tracing performance by reducing intersection tests.
- Isotropy: A constraint to keep base mesh triangles roughly equilateral, which is optimal for hardware subdivision.
- Visual-Guided Adaptive LOD: An algorithm that uses image-space error to decide which parts of the mesh need more subdivision, allowing for more detail where it is visually necessary.
Evaluation and Results
The researchers evaluated their method on a dataset of 121 high-resolution meshes (ranging from 0.1 million to 4 million faces). The proposed method consistently outperformed previous techniques across all categories:
- Higher Quality: On "Extreme" meshes (>2M faces), the method achieved a rendering PSNR of 44.89, compared to 42.61 for the previous state-of-the-art.
- Better Efficiency: It produced significantly smaller shell volumes (0.083 vs 0.089), translating to faster ray-tracing performance on modern GPU architectures like NVIDIA's Ada Lovelace.
- Robustness: Thanks to the Laplacian reparameterization, the base meshes maintained better isotropy (triangle quality) even after heavy optimization, avoiding the artifacts common in staged pipelines.
Discussion and Future Directions
While the results are impressive, the authors note that the iterative optimization process takes longer than simple non-differentiable baking (roughly 4-5 minutes for extreme meshes). Future work could integrate more complex material models (BRDFs) and textures into the differentiable loop to handle non-diffuse surfaces more accurately. Additionally, further tuning for specific hardware-level compression formats could yield even smaller file sizes, pushing the boundaries of real-time extreme geometry rendering.
Summary updated