Towards Zero-Shot Point Cloud Registration Across Diverse Scales, Scenes, and Sensor Setups

Hyungtae Lim, Minkyun Seo, Luca Carlone, Jaesik ParkarXiv 2026 · arXiv · OpenReview

TL;DR

This paper introduces BUFFER-X, a novel framework for zero-shot point cloud registration that effectively generalizes across diverse scales, scenes, and sensor types without requiring any manual tuning or retraining. By addressing key limitations of existing methods, BUFFER-X achieves robust registration performance on a challenging new benchmark, with an efficient variant, BUFFER-X-Lite, offering practical speed-accuracy trade-offs.

What Problem Does This Paper Address?

Accurately aligning 3D point clouds (point cloud registration) is crucial for applications like robotics, augmented reality, and autonomous driving. While deep learning methods have shown great promise, they often struggle to generalize to new environments or sensor setups they haven't been trained on. This "zero-shot" generalization problem is a major hurdle for real-world deployment. Existing methods typically require dataset-specific hyperparameter tuning (like voxel size or search radius) or retraining, which is impractical. The authors identify three core reasons for this: 1) fixed parameters that don't adapt to varying scales and densities, 2) learned keypoint detectors that fail to transfer across domains, and 3) the amplification of scale differences by using absolute coordinates.

Methods

BUFFER-X tackles these issues with a three-pronged, training-free approach:

  1. Geometric Bootstrapping: Instead of fixed parameters, BUFFER-X automatically estimates optimal voxel sizes and search radii for each point cloud pair. It uses principal component analysis (PCA) on sampled points to understand their spatial distribution (sphericity) and point density to adapt these parameters.
  2. Distribution-Aware Farthest Point Sampling (FPS): To overcome the poor transferability of learned keypoint detectors, BUFFER-X replaces them with FPS. This method samples points based on their distribution, ensuring robustness.
  3. Patch-Level Coordinate Normalization & Multi-Scale Matching: To handle scale mismatches, BUFFER-X normalizes point coordinates within local patches to a consistent range ([-1, 1]). It then employs a hierarchical multi-scale matching strategy, extracting features and correspondences at local, middle, and global receptive fields. This allows for robust matching even when features at one scale might be insufficient.

For efficiency, BUFFER-X-Lite introduces two optimizations: adaptive early exit (stopping processing if sufficient inliers are found at an earlier scale) and a faster pose solver (KISS-Matcher) to reduce computation time by up to 43% while maintaining accuracy.

Datasets

The authors introduce a comprehensive benchmark for evaluating zero-shot generalization, comprising 12 diverse datasets:

  • Object-scale: ModelNet40
  • Indoor: 3DMatch, 3DLoMatch, ScanNet++i, ScanNet++F
  • Outdoor: TIERS, ETH, KITTI, WOD, KAIST, MIT, Oxford

This benchmark specifically includes scenarios with heterogeneous LiDAR sensor configurations, where source and target point clouds are captured by different sensor types (e.g., Velodyne VLP-16 to Ouster OS1-64), pushing the boundaries of cross-domain evaluation.

Key Insights and Ideas

  • Parameter-free is Key: The core insight is that fixed, manually tuned parameters are a major bottleneck for generalization. Automatically adapting these parameters based on the input data's geometry is crucial.
  • FPS as a Robust Detector: Replacing learned keypoint detectors with a non-learning-based method like FPS significantly improves cross-domain transferability.
  • Scale Normalization is Paramount: Normalizing point coordinates within local patches is essential for making features domain-invariant and handling large scale discrepancies.
  • Multi-Scale Synergy: Combining local, middle, and global receptive fields for feature extraction and matching is more powerful than relying on a single scale, as different scales capture complementary information.
  • Early Exit for Efficiency: Not all registration problems are equally difficult. An adaptive early exit strategy can significantly speed up processing by skipping unnecessary steps for easier cases.

Key Results

  • BUFFER-X achieves state-of-the-art zero-shot generalization performance across the 12 diverse datasets, significantly outperforming existing methods, especially when no manual tuning is allowed.
  • On the heterogeneous LiDAR benchmark, BUFFER-X demonstrates strong sensor-agnostic generalization, achieving the highest success rates without manual tuning, even for challenging sparse-to-dense scenarios.
  • BUFFER-X-Lite reduces computation time by 43% on average while maintaining comparable accuracy.
  • The KISS-Matcher solver, used in BUFFER-X-Lite, is up to 100x faster than RANSAC for pose estimation, particularly on LiDAR data.
  • Qualitative results show BUFFER-X producing visually accurate registrations across various challenging scenarios, including significant scale differences and sensor variations.

Limitations and Future Work

  • Partial Overlap: The paper acknowledges that BUFFER-X shows a slightly lower success rate on datasets with very low overlap (10-30%, like 3DLoMatch). This is because the method prioritizes maximizing the number of correspondences, which might not always yield the optimal pose in extreme partial overlap scenarios. This highlights a trade-off between generalization and robustness to partial overlaps.
  • Future Work: The authors suggest exploring the integration of semantic cues to further improve robustness in extremely low-overlap scenarios. They also aim to enhance usability through simplified installation and ROS integration.

Summary updated

Abstract

Some deep learning-based point cloud registration methods struggle with zero-shot generalization, often requiring dataset-specific hyperparameter tuning or retraining for new environments. We identify three critical limitations: (a) fixed user-defined parameters (e.g., voxel size, search radius) that fail to generalize across varying scales, (b) learned keypoint detectors exhibit poor cross-domain transferability, and (c) absolute coordinates amplify scale mismatches between datasets. To address these three issues, we present BUFFER-X, a training-free registration framework that achieves zero-shot generalization through: (a) geometric bootstrapping for automatic hyperparameter estimation, (b) distribution-aware farthest point sampling to replace learned detectors, and (c) patch-level coordinate normalization to ensure scale consistency. Our approach employs hierarchical multi-scale matching to extract correspondences across local, middle, and global receptive fields, enabling robust registration in diverse environments. For efficiency-critical applications, we introduce BUFFER-X-Lite, which reduces total computation time by 43% (relative to BUFFER-X) through early exit strategies and fast pose solvers while preserving accuracy. We evaluate on a comprehensive benchmark comprising 12 datasets spanning object-scale, indoor, and outdoor scenes, including cross-sensor registration between heterogeneous LiDAR configurations. Results demonstrate that our approach generalizes effectively without manual tuning or prior knowledge of test domains. Code: https://github.com/MIT-SPARK/BUFFER-X.
BibTeX
@misc{lim2026towards,
  author = {Hyungtae Lim and Minkyun Seo and Luca Carlone and Jaesik Park},
  title = {{Towards Zero-Shot Point Cloud Registration Across Diverse Scales, Scenes, and Sensor Setups}},
  year = {2026},
  eprint = {2601.02759},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.02759},
}
Figure