Registration of Multi-View Point Sets Under the Perspective of Expectation-Maximization

Jihua Zhu, Rui Guo, Zhongyu Li, Jing Zhang, Shanmin PangIEEE TIP 2020 · OpenReview

TL;DR

This paper introduces a novel approach to multi-view point set registration, a crucial step for 3D reconstruction. By framing the problem as a maximum likelihood estimation task and leveraging the Expectation-Maximization (EM) algorithm, their method effectively aligns multiple 3D scans from different viewpoints, achieving high accuracy and efficiency.

What Problem Does This Paper Address?

Creating a complete 3D model of an object or scene often requires capturing it from multiple angles using 3D scanners. However, each scan only captures a portion of the object, and these individual "point sets" are in their own coordinate systems. To build a unified 3D model, these point sets must be accurately aligned – a process called multi-view registration. Existing methods often struggle with either not using all available information effectively or introducing unnecessary computational complexity. Some methods are prone to local minima or cannot handle non-overlapping regions, hindering robust and accurate 3D reconstruction. This paper aims to address these limitations by proposing a more principled and efficient EM-based approach.

Methods

The core of this paper's contribution lies in its novel formulation of the multi-view registration problem. Instead of traditional pairwise registration, they treat the entire multi-view registration as a single maximum likelihood estimation problem. The key idea is to model each data point in a point set as being generated by a unique Gaussian Mixture Model (GMM). Crucially, the centroids of these GMMs are not directly observed but are approximated by the nearest neighbors of the data point in other well-aligned point sets. This assumption allows them to define a likelihood function that incorporates all rigid transformations (rotations and translations) for each point set.

The Expectation-Maximization (EM) algorithm is then employed to iteratively estimate these transformations and a single GMM covariance. The EM algorithm consists of two steps:

  • E-step (Expectation): This step involves finding the nearest neighbors for each point in other point sets to approximate the GMM centroids. It then calculates the posterior probability of a data point being generated by each of these GMM components.
  • M-step (Maximization): This step uses the probabilities calculated in the E-step to update the rigid transformations (rotations and translations) for each point set and the GMM covariance.

This approach automatically determines the number of GMM components based on the number of point sets, avoiding the need for manual parameter tuning for the number of clusters, which is a common issue in other GMM-based methods.

Datasets

The authors evaluated their proposed method, EMPMR (Expectation-Maximization Perspective for Multi-View Registration), on six benchmark datasets. Four of these datasets were sourced from the Stanford 3D Scanning Repository, and two were provided by Torsello. These datasets consist of multiple point sets of objects acquired from different viewpoints, along with ground truth rigid transformations for registration. For experimental purposes, the data was down-sampled to approximately 2000 points per set to manage runtime.

Key Insights and Ideas

  • Unified Multi-View Registration: Treating multi-view registration as a single maximum likelihood problem, rather than a series of pairwise registrations, allows for a more holistic optimization.
  • Nearest Neighbors as GMM Centroids: Using nearest neighbors from other point sets as approximate GMM centroids is an ingenious way to handle the lack of direct correspondences, a common challenge in registration.
  • Automatic GMM Component Determination: By linking GMM components to the number of input point sets, the method bypasses the need to pre-specify the number of clusters, a significant advantage over other GMM-based techniques.
  • Robustness through EM: The iterative nature of the EM algorithm, combined with the probabilistic modeling, inherently provides robustness to noise and outliers.

Key Results

The proposed EMPMR method demonstrated superior performance across the tested datasets.

  • Accuracy: EMPMR consistently achieved the most accurate registration results, with low rotation and translation errors, outperforming methods like K-means, MATrICP, and JRMPC on most datasets.
  • Efficiency: EMPMR's runtime was found to be comparable to the efficient K-means method and significantly faster than JRMPC, making it a practical choice for complex 3D reconstruction tasks.
  • Robustness: The method showed strong robustness to noise, maintaining high accuracy even when datasets were perturbed with Gaussian noise, outperforming other methods in these challenging scenarios.
  • Scene Reconstruction: The paper also demonstrated the applicability of EMPMR to 3D scene reconstruction using RGB-D data, showcasing its potential for real-world applications.

Limitations and Future Work

  • Local Convergence: Like many optimization-based methods, EMPMR is a local convergent algorithm. The paper acknowledges that good initial guesses for rigid transformations are necessary for accurate registration.
  • Parameter Sensitivity: While the paper investigates the sensitivity to the outlier parameter w, it suggests that EMPMR is relatively insensitive as long as w is chosen within a suitable range (0.0005-0.05), simplifying its application.
  • Future Work: The authors suggest extending their approach to solve the problem of robot 3D mapping, indicating a broader applicability beyond static scene reconstruction.

Summary updated

Abstract

Multi-view registration plays a critical role in 3D model reconstruction. To solve this problem, most previous methods align point sets by either partially exploring available information or blindly utilizing unnecessary information, which may lead to undesired results or extra computation complexity. Accordingly, we propose a novel solution for the multi-view registration under the perspective of Expectation-Maximization (EM). The proposed method assumes that each data point is generated from one unique Gaussian Mixture Model (GMM), where its corresponding points in other point sets are regarded as Gaussian centroids with equal covariance and membership probabilities. As it is difficult to obtain real corresponding points in the registration problem, they are approximated by the nearest neighbor in each other aligned point sets. Based on this assumption, it is reasonable to define the likelihood function including all rigid transformations, which require to be estimated for multi-view registration. Subsequently, the EM algorithm is derived to estimate rigid transformations with one Gaussian covariance by maximizing the likelihood function. Since the GMM component number is automatically determined by the number of point sets, there is no trade-off between registration accuracy and efficiency in the proposed method. Finally, the proposed method is tested on several benchmark data sets and compared with state-of-the-art algorithms. Experimental results demonstrate its superior performance on the accuracy, efficiency, and robustness for multi-view registration.
BibTeX
@article{zhu2020registration,
  author = {Jihua Zhu and Rui Guo and Zhongyu Li and Jing Zhang and Shanmin Pang},
  title = {{Registration of Multi-View Point Sets Under the Perspective of Expectation-Maximization}},
  journal = {IEEE Trans. Image Process.},
  year = {2020},
  url = {https://openreview.net/forum?id=0GTxAdtv7o},
}
Figure