Abstract: Maps are arguably one of the most fundamental concepts used to define and operate on manifold surfaces in differentiable geometry. Accordingly, in geometry processing, maps are ubiquitous and are used in many core applications, such as paramterization, shape analysis, remeshing, and deformation. Unfortunately, most computational representations of surface maps do not lend themselves to manipulation and optimization, usually entailing hard, discrete problems. While algorithms exist to solve these problems, they are problem-specific, and a general framework for surface maps is still in need. In this paper, we advocate considering neural networks as encoding surface maps. Since neural networks can be composed on one another and are differentiable, we show it is easy to use them to define surfaces via atlases, compose them for surface-to-surface mappings, and optimize differentiable objectives relating to them, such as any notion of distortion, in a trivial manner. In our experiments, we represent surfaces by generating a neural map that approximates a UV parameterization of a 3D model. Then, we compose this map with other neural maps which we optimize with respect to distortion measures. We show that our formulation enables trivial optimization of rather elusive mapping tasks, such as maps between a collection of surfaces.
Introduction
In 3D geometry processing, "maps" are essential for projecting 2D textures onto 3D models (parameterization) or establishing correspondences between different shapes (surface-to-surface mapping). Traditionally, these maps are defined over triangular meshes, leading to complex combinatorial optimization problems. When a point moves across a mesh edge, the underlying mathematical representation changes abruptly, making optimization non-smooth and difficult for complex shapes. Most existing solutions are highly specialized and lack a general framework.
Methodology
The authors introduce the Neural Surface Map framework, which reimagines geometry processing by replacing discrete mesh mappings with smooth, differentiable neural networks. The approach consists of three primary components:
- Neural Overfitting: Instead of learning general shape categories, a small Multi-Layer Perceptron (MLP), $\phi: \mathbb{R}^2 \to \mathbb{R}^3$, is "overfitted" to a specific 3D model. By training the network to mimic a standard UV parameterization, the weights memorize the geometry, providing a continuous surface representation.
- Compositional Algebra: To map Surface A to Surface B, an "inner" neural map $h: \mathbb{R}^2 \to \mathbb{R}^2$ is introduced. The complete surface-to-surface map $f$ is defined as the composition $\psi \circ h \circ \phi^{-1}$, where $\phi$ and $\psi$ represent the source and target surfaces respectively.
- Differentiable Optimization: Since the pipeline is entirely composed of neural networks, geometric objectives—such as minimizing Symmetric Dirichlet energy (distortion)—can be formulated as loss functions. Optimization is performed via backpropagation through the composition to update the weights of $h$.
Experiments and Results
The framework was evaluated on standard benchmarks including the Stanford Bunny, FAUST, TOSCA, and various complex sculptures.
High-Fidelity Reconstruction
The overfitted networks demonstrate the ability to capture extremely fine geometric details with minimal deviation from ground-truth meshes. Error maps across various models show consistently low reconstruction error.
Surface Mapping Performance
The method excels at "elusive" mapping tasks, such as establishing correspondences between highly non-isometric shapes like a hippo and a cow. Using only four keypoint constraints, the method achieved an average Symmetric Dirichlet energy of 11. Notably, the neural approach produces significantly smoother maps with fewer "triangle flips" (overlaps) compared to state-of-the-art discrete methods. For instance, on animal models, the neural map exhibited only 0.03% flips, whereas some baseline configurations reached nearly 50%.
Discussion and Conclusion
Neural Surface Maps offer several advantages:
- Resolution Independence: Optimization cost scales with the number of sample points rather than mesh triangle count.
- Inherent Cycle Consistency: Using a common 2D domain for a collection of shapes ensures consistent mappings across the group.
- Analytic Flexibility: The framework easily maps complex meshes to analytic surfaces like Mobius strips or tori.
Current limitations include a requirement for "disk-topology" surfaces, necessitating manual preprocessing for shapes with holes or handles. While bijectivity is encouraged through loss functions, it is not mathematically guaranteed. Future work may involve solving Partial Differential Equations (PDEs) directly on these neural surfaces or integrating the framework into broader deep learning pipelines for shape analysis.
For more information, visit the project page.
Summary updated