Pose Synchronization under Multiple Pair-wise Relative Poses

Yifan Sun, Qixing HuangCVPR 2023 · DOI

Abstract

Pose synchronization is a critical component of 3D reconstruction and multi-view geometry. However, standard methods often fail in "extreme" scenarios where pairwise matching produces multiple competing hypotheses, many of which are incorrect. This paper introduces a robust three-step framework that utilizes iterative diffusion-based candidate generation and Markov Random Field (MRF) selection to achieve state-of-the-art accuracy in high-outlier, multi-hypothesis environments.

Problem Statement

Pose synchronization involves taking a collection of noisy relative poses (the orientation and position of object A relative to object B) and estimating a consistent set of absolute poses in a global coordinate system. While foundational to robotics and 3D scanning, real-world conditions—such as repeating symmetries or low image overlap—frequently lead to multiple relative pose estimates per pair.

Overview of pose synchronization with multiple relative pose hypotheses for 3D reconstruction.

Most existing techniques assume a single relative pose estimate per pair and a high ratio of "inliers" (correct matches). In practice, the majority of these estimates may be wrong. Standard low-rank matrix recovery methods typically collapse under these high-outlier conditions.

Methodology

The proposed framework addresses these challenges through a three-stage pipeline:

  1. Candidate Pose Generation (Diffusion & Clustering): The algorithm leverages the principle that correct relative poses are consistent across multiple paths in an observation graph. By "diffusing" poses along paths and using a continuous Gaussian Mixture Model (GMM) to cluster propagated results, the system identifies likely absolute pose candidates without the discretization errors found in grid-based methods.
  2. Optimal Pose Selection (MRF Inference): Once candidates are generated, the problem is framed as a Markov Random Field (MRF) task: selecting the combination of candidates that is globally most consistent. This is solved efficiently using a "projected power method," an iterative linear algebra approach that avoids the complexity of traditional MRF solvers.
  3. Pose Refinement: The resulting global "skeleton" is fine-tuned using a robust continuous optimization (Gauss-Newton Method) under a Geman-McClure loss function to eliminate residual noise.

Key Insights

  • Truth Propagates, Noise Dissipates: By composing relative poses along paths (e.g., $A \to B \to C$), true signals reinforce one another while random outliers tend to cancel out. This allows the system to recover the correct pose even when direct pairwise matches are erroneous.
  • Continuous Formulation: By avoiding the "discretization trap" of dividing the search space into bins, the GMM-based clustering maintains high precision throughout the pipeline.
  • Value in Ambiguity: Unlike "top-1" methods that discard all but the best match, this framework treats secondary and tertiary matches as valuable data, which often contain the ground truth in ambiguous scenes.

Experimental Results

The approach was evaluated on synthetic datasets and large-scale real-world benchmarks including ScanNet and Cornell-Artquad.

  • Robustness: On synthetic datasets with up to 80% incorrect poses, the method maintained nearly 90% accuracy, whereas baselines like SE-Sync or IRLS failed completely.
  • Real-World Performance: On the ScanNet (RGB-D) dataset, the method achieved a rotation accuracy of 56.4% (within a 10° threshold) when using four relative pose candidates per pair. This outperformed the next best baseline, K-Best, by over 17%.
  • Efficiency: The framework is computationally efficient, processing the San-Francisco dataset (7,866 images) in approximately 263 seconds.

Conclusion

This research demonstrates that leveraging multiple hypotheses rather than discarding them leads to significantly more robust 3D reconstructions in challenging environments. Future work will focus on unifying the candidate generation and selection steps into a single diffusion-based optimization to further improve performance and reduce the computational bottleneck of clustering.

Summary updated

BibTeX
@inproceedings{sun2023pose,
  author = {Yifan Sun and Qixing Huang},
  title = {{Pose Synchronization under Multiple Pair-wise Relative Poses}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.01256},
}
Figure