Topology-Aware Non-Rigid Point Cloud Registration

Konstantinos Zampogiannis, Cornelia Fermüller, Yiannis Aloimonos

Abstract

Non-rigid point cloud registration often fails during topological changes, such as objects separating, due to the "sticky" nature of spatial smoothness constraints. We present FB-Warp, a pipeline that leverages time symmetry to detect topological events and blend motion hypotheses. By identifying regions of stretching and compression between forward and backward warps, our method achieves state-of-the-art accuracy on dynamic scenes, reducing error by 30% at separation boundaries.

Introduction

Non-rigid registration is a foundational task for 3D reconstruction and robotics, yet it consistently struggles with "topology changes"—events where surfaces touch or separate. Standard algorithms typically rely on $k$-nearest neighbor ($k$-NN) graphs to ensure motion smoothness. However, when objects move apart, these graphs often "stick" the surfaces together, causing the motion estimate to smear across the newly formed gap.

RGB-D input and corresponding point cloud representation of a hand-cup interaction.

While volumetric approaches using Signed Distance Fields can mitigate this, they are computationally expensive and do not scale well. We propose a topology-aware approach that operates directly on unorganized point clouds, ensuring efficiency and scalability for large-scale scenes.

Methodology: The FB-Warp Pipeline

The core innovation of our approach, FB-Warp, is a post-processing phase that utilizes the symmetry of forward and backward motion to resolve topological ambiguities.

  1. Dual Warp Hypotheses: We compute both a standard forward warp ($S \to D$) and a backward warp ($D \to S$). A "separation" in forward time is a "contact" in backward time. Because standard algorithms handle contacts effectively (as no gap exists to bridge), the inverted backward warp provides a "clean" hypothesis at separation boundaries.
  2. Event Detection: The algorithm identifies topological events by analyzing local "neighborhood stretching" and "compression." Significant stretching in the forward warp indicates a separation, while compression indicates a contact.
  3. Local Blending: We use a Radial Basis Function (RBF) kernel to smoothly blend the two hypotheses. The system trusts the inverted backward warp near separations and the standard forward warp near contacts or stable regions.
Visualization of the FB-Warp pipeline, showing event detection (center) and motion hypothesis blending.

Results and Evaluation

We evaluated FB-Warp on the MPI Sintel dataset and a custom "Topological Event" dataset featuring manual annotations for contacts and separations (e.g., hand-clapping, opening drawers).

  • State-of-the-Art Accuracy: FB-Warp achieved the lowest overall Endpoint Error (EPE) on the MPI Sintel dataset compared to baselines like PD-Flow.
  • Significant Error Reduction: In regions undergoing "close-to-open" (separation) changes, the method reduced registration error by approximately 30% compared to topology-agnostic baselines.
  • Robust Event Detection: The system achieved an F-score of 0.8 in explicitly detecting topological events, proving its utility for higher-level scene understanding.

Conclusion

FB-Warp effectively addresses the "sticky" boundary problem in non-rigid registration by exploiting time symmetry. While challenges remain with sliding contacts where local distances do not change significantly, the method provides a robust, topology-aware framework for dynamic 3D reconstruction without requiring pre-existing models or semantic segmentation.

Summary updated

Abstract

In this paper, we introduce a non-rigid registration pipeline for pairs of unorganized point clouds that may be topologically different. Standard warp field estimation algorithms, even under robust, discontinuity-preserving regularization, tend to produce erratic motion estimates on boundaries associated with `close-to-open' topology changes. We overcome this limitation by exploiting backward motion: in the opposite motion direction, a `close-to-open' event becomes `open-to-close', which is by default handled correctly. At the core of our approach lies a general, topology-agnostic warp field estimation algorithm, similar to those employed in recently introduced dynamic reconstruction systems from RGB-D input. We improve motion estimation on boundaries associated with topology changes in an efficient post-processing phase. Based on both forward and (inverted) backward warp hypotheses, we explicitly detect regions of the deformed geometry that undergo topological changes by means of local deformation criteria and broadly classify them as `contacts' or `separations'. Subsequently, the two motion hypotheses are seamlessly blended on a local basis, according to the type and proximity of detected events. Our method achieves state-of-the-art motion estimation accuracy on the MPI Sintel dataset. Experiments on a custom dataset with topological event annotations demonstrate the effectiveness of our pipeline in estimating motion on event boundaries, as well as promising performance in explicit topological event detection.
BibTeX
@article{zampogiannis2018topology,
  author = {Konstantinos Zampogiannis and Cornelia Fermüller and Yiannis Aloimonos},
  title = {{Topology-Aware Non-Rigid Point Cloud Registration}},
  journal = {IEEE Trans. Pattern Anal. Mach. Intell.},
  year = {2018},
  doi = {10.1109/tpami.2019.2940655},
}
Figure