TL;DR
This paper introduces Implicit Displacement Fields (IDF), a novel 3D representation that decomposes complex shapes into a smooth base surface and a high-frequency displacement field. By leveraging the frequency biases of neural networks, IDF achieves unsupervised disentanglement of geometric details, leading to superior reconstruction quality, better training stability, and the ability to transfer fine details between different 3D models.
What Problem Does This Paper Address?
Neural implicit functions (like Signed Distance Functions or SDFs) have become a go-to for continuous 3D representation. However, they face a "spectral bias" dilemma: standard networks naturally learn low-frequency, smooth shapes first. To capture fine details (high-frequency signals), researchers often turn to spatial partitioning (which consumes massive memory) or specialized architectures like SIRENs. While SIRENs can represent details, they are notoriously difficult to train, often prone to overfitting or getting stuck in poor local minima.
The authors seek a middle ground: a representation that is as detailed as a high-frequency SIREN but as stable and easy to optimize as a smooth, low-frequency network.
Methods
The core idea is inspired by displacement mapping, a classic computer graphics technique where a detailed surface is created by offsetting a smooth base surface along its normals (see Figure 1).
The authors translate this into a neural framework using two SIREN networks:
- Base Network ($N^{\omega_B}$): A low-frequency network that represents a smooth "base" SDF.
- Displacement Network ($N^{\omega_D}$): A high-frequency network that predicts the scalar displacement distance along the base surface's normal.
Key Technical Innovations: * Inverse Displacement Formulation: To make training efficient, they use an "inverse" displacement field. This allows them to pre-sample points on the ground-truth detailed surface and map them back to the base surface, avoiding costly dynamic sampling during optimization. * Progressive Training: They employ a coarse-to-fine training strategy, first fitting the smooth base shape and then gradually "turning on" the displacement network. * Transferable IDF: By replacing raw 3D coordinates with scale-and-translation invariant features relative to the base surface (using a global context descriptor and FiLM conditioning), they allow the learned displacement "texture" to be applied to entirely new base shapes.
Datasets
The authors evaluated their method on:
- Sketchfab-16: A set of 14 high-resolution 3D models from Sketchfab and 2 from the Stanford 3DScanRepo.
- Detail Transfer Sets: Shapes provided by Berkiten et al. (2017), Yang et al. (2020), and Zhou & Jacobson (2016) were used to test how well details (like clothing wrinkles or facial features) could be transplanted.
Key Insights and Ideas
- Frequency Disentanglement by Construction: By simply setting different frequency hyperparameters ($\omega$) for the two networks, the model naturally partitions the geometry into "structure" and "detail" without needing explicit labels.
- Geometric Constraints as Regularizers: Forcing the high-frequency signal to follow the normal of a smooth base surface acts as a powerful geometric prior. This prevents the "noisy" artifacts often seen when training high-frequency networks from scratch.
- Relative vs. Absolute Coordinates: The shift from absolute $XYZ$ coordinates to relative features (distance to surface, local context) is what unlocks transferability, effectively creating a "neural height map" that doesn't care about the underlying global pose.
Key Results
- State-of-the-Art Fidelity: IDF outperformed all baselines in reconstruction accuracy. On the Sketchfab-16 dataset, it achieved a Chamfer distance of $1.22 \times 10^{-3}$, significantly lower than FFN ($5.47$) or D-SDF ($2.85$).
- Efficiency: While NGLOD (a spatial partitioning method) achieved comparable results at its highest level of detail, it required 256x more parameters and nearly 200x more memory (946MB vs. 4.8MB for IDF).
- Stability: Unlike standard high-frequency SIRENs, which often diverged during training, IDF remained stable across all tested models.
- Detail Transfer: The method successfully transferred complex, spatially-varying details (like eyebrows or fabric folds) to new target shapes without requiring any correspondence mapping between the source and target.
Limitations and Future Work
- Pre-alignment Requirement: For detail transfer, the source and target shapes currently need to be roughly aligned. The authors suggest exploring sparse correspondences in the future to automate this alignment.
- Sparse Data Sensitivity: Stress tests showed that when training data is extremely sparse (e.g., only 1% of points), the model can overfit and generate high-frequency noise.
- Inference Speed: While memory-efficient, the current implementation's inference time is slightly slower than some optimized spatial structures like NGLOD, leaving room for performance optimization.
Summary updated