Abstract: Maps are arguably one of the most fundamental concepts used to define and operate on manifold surfaces in differentiable geometry. Accordingly, in geometry processing, maps are ubiquitous and are used in many core applications, such as parameterization, shape analysis, remeshing, and deformation. Unfortunately, most computational representations of surface maps do not lend themselves to manipulation and optimization, usually entailing hard, discrete problems. While algorithms exist to solve these problems, they are problem-specific, and a general framework for surface maps is still in need. In this paper, we advocate to consider neural networks as encoding surface maps. Since neural networks can be composed on one another and are differentiable, we show it is easy to use them to define surfaces via atlases, compose them for surface-to-surface mappings, and optimize differentiable objectives relating to them, such as any notion of distortion, in a trivial manner. In our experiments, we represent surfaces by generating a neural map that approximates a UV parameterization of a 3D model. Then, we compose this map with other neural maps which we optimize with respect to distortion measures. We show that our formulation enables trivial optimization of rather elusive mapping tasks, such as maps between a collection of surfaces.
Introduction
In classical geometry processing, surfaces are traditionally represented as meshes—collections of vertices and triangles. While intuitive, meshes present significant challenges for optimization. Tasks such as finding a smooth map between two different 3D models (e.g., mapping a hippo's skin onto a cow) require complex discrete bookkeeping to track point locations and prevent triangle flips. These combinatorial hurdles make UV parameterization, shape correspondence, and cycle-consistency extremely complex and problem-specific. Neural Surface Maps (NSM) address this by replacing these "hard" discrete problems with "soft" differentiable ones, representing 3D surfaces and their mappings as neural networks rather than discrete meshes.
Methodology
The core methodology of NSM treats a surface map as a continuous function $\phi: \mathbb{R}^2 \to \mathbb{R}^3$. The process involves three main components:
- Neural Surface Representation: A neural network (MLP) is "overfitted" to a specific 3D mesh, learning to map 2D UV coordinates to 3D surface points. This makes the network itself the representation of the surface.
- Algebra of Composition: Since these maps are functions, they can be composed. A mapping between two surfaces $\mathcal{S}$ and $\mathcal{T}$ is defined by an "inner" neural map $h: \mathbb{R}^2 \to \mathbb{R}^2$ operating in the 2D domain. The full surface-to-surface map is then $f = \psi \circ h \circ \phi^{-1}$.
- Differentiable Distortion: Distortion measures, such as Symmetric Dirichlet energy, are calculated directly from the network's Jacobians. Using automatic differentiation in frameworks like PyTorch, these measures can be minimized to ensure the map is bijective and low-distortion.
By overfitting to single shapes, NSM captures geometry in a continuous, differentiable container, decoupling the quality of the map from the original mesh resolution. Furthermore, mapping a collection of shapes to a common 2D domain guarantees cycle-consistency by construction.
Results and Evaluation
The authors evaluated NSM on standard benchmarks including the Stanford Bunny, the Bimba bust, and datasets like FAUST and Tosca.
Key results include:
- Superior Smoothness: NSM produces maps without "triangle flips," achieving 0% flips in animal mapping tests where traditional baselines often reached 50%.
- Low Distortion: On the Stanford Bunny, the method significantly improved the isometry of the parameterization, reducing median Dirichlet energy from 7.24 to 6.03.
- Extreme Remapping: The system successfully performed texture transfer between highly dissimilar shapes, such as a hippo and a cow, using only minimal keypoint constraints.
Conclusion and Future Work
While NSM simplifies many geometry processing tasks, it currently faces limitations regarding topology; surfaces must be cut into disk-like patches to be flattened. Additionally, the optimization time for overfitting a single model (3 to 7 hours) is slower than traditional linear methods. Future research may explore applying differential geometry operators directly to these neural surfaces to solve partial differential equations (PDEs) or utilizing this representation for downstream tasks like 3D shape segmentation and classification.
Summary updated