TL;DR
Neural Deformation Pyramid (NDP) introduces a hierarchical approach to non-rigid point cloud registration by decomposing complex motion into a sequence of levels. By using a pyramid of coordinate-based MLPs with increasing sinusoidal frequencies, the method achieves state-of-the-art accuracy while running 50 times faster than previous MLP-based optimization techniques.
What Problem Does This Paper Address?
Point cloud registration is the process of aligning two sets of 3D points. While "rigid" registration (like moving a static chair) only requires finding a single rotation and translation, non-rigid registration (like a person walking or a flag waving) is significantly harder. The algorithm must estimate a unique movement for every single point in the cloud.
This task is plagued by three major challenges:
- High Dimensionality: There are effectively infinite ways a shape can deform.
- Sensor Noise and Occlusions: Real-world data is messy, often missing large chunks of the object (partial-to-partial registration).
- Computational Cost: Previous "coordinate-MLP" methods (which represent motion as a continuous function) were notoriously slow to converge because they tried to learn all scales of motion simultaneously.
Methods
The core innovation is the Neural Deformation Pyramid (NDP). Instead of one giant network, the authors use a sequence of small Multi-Layer Perceptrons (MLPs).
- Hierarchical Decomposition: The motion is broken down from global to local. Think of an articulating figure, like the dinosaur shown below: Level 1 handles the overall rigid alignment; Level 4 handles the articulation of limbs; and Level 8 handles the fine, non-rigid surface details.
- Sinusoidal Frequency Control: Each level of the pyramid uses a "positional encoding" (similar to those in Transformers or NeRF) but with a twist. Higher levels in the pyramid use low-frequency sinusoids, which naturally constrain the network to produce smooth, rigid-like motions. As you go deeper into the pyramid, the frequency increases, allowing the network to represent increasingly complex, non-rigid details.
- Recursive Increments: Each level takes the output of the previous level and adds a "motion increment."
- Deformability Mask: The network also outputs a confidence score ($\alpha$) that acts as a learned regulator, telling the system how much "non-rigidity" to allow at that specific level and location.
- Optimization: The model can be used in a "no-learned" (unsupervised) mode using Chamfer distance or a "supervised" mode (LNDP) using learned correspondences.
Datasets
The authors evaluated their method on the 4DMatch and 4DLoMatch benchmarks.
- These datasets are derived from DeformingThings4D, a collection of high-quality animations of animals and humans.
- 4DMatch features high overlap (45–92%), while 4DLoMatch is much harder, featuring low overlap (15–45%) and significant occlusions, mimicking real-world sensor limitations.
Key Insights and Ideas
- Frequency as a Regularizer: One of the most elegant insights is that the frequency of the input encoding acts as a natural bottleneck for rigidity. You don't need complex manual constraints if the math of the encoding physically prevents the network from "wiggling" too much at the top levels.
- Efficiency through Specialization: By forcing each MLP to only care about a specific frequency band, the optimization becomes much more stable. This is why they achieve a 50x speedup over methods like Nerfies—the optimizer isn't fighting to balance global and local scales at the same time.
- $L_1$ over $L_2$: The authors found that using the $L_1$ norm for their distance loss was crucial for partial registration. $L_2$ (squaring the error) over-penalizes points that don't have a match in the other cloud, whereas $L_1$ is more "forgiving" of these outliers.
Key Results
- Speed: NDP converges in roughly 738 iterations across all levels, compared to nearly 3,800 iterations for a single-scale MLP approach.
- Accuracy: On the 4DMatch benchmark, the supervised version (LNDP) achieved a 75.26% Registration Recall (AccR), significantly outperforming previous SOTA methods like Lepard+NICP (65.32%).
- Robustness: Even in the "no-learned" setting (where the model hasn't seen the data before), NDP outperformed several supervised baselines on the 4DMatch benchmark.
- Qualitative Success: The method successfully performs "shape transfer," mapping the motion of a human model onto a stylized character while preserving the character's unique geometry.
Limitations and Future Work
- Topological Changes: Because NDP operates in 3D Euclidean space, it struggles with "topological" changes—for example, if a character's hands clap together and merge into one surface, the model may struggle to separate them.
- Non-Isometric Cases: The current unsupervised version assumes the surface area/distance between points stays relatively constant (isometry). It can fail on extreme stretches (e.g., mapping a chest to a belly).
- Real-time Constraints: While 50x faster than previous MLPs, it is not yet "real-time" for high-resolution clouds, though the authors suggest custom CUDA kernels could bridge this gap.
Summary updated