Abstract
This paper proposes a robust method for registering Mobile Mapping System (MMS) LiDAR point clouds with panoramic image sequences using a Relative Orientation Model (ROM). By leveraging the geometric relationships between consecutive panoramic images, the method solves for relative position and attitude parameters, enabling accurate data fusion even in environments where GPS/IMU data may be unreliable. The approach utilizes SURF for feature matching and RANSAC for outlier removal. Experimental results across two distinct datasets demonstrate high registration accuracy, with mean errors of 7.36 pixels and 3.75 pixels, respectively.
Introduction
Mobile Mapping Systems (MMS) are essential for creating high-definition maps for autonomous driving and urban modeling. These systems simultaneously capture 3D LiDAR points and 2D panoramic images. However, accurately fusing these data types—ensuring that every 3D point corresponds correctly to its 2D image pixel—remains a significant challenge.
Existing registration methods often suffer from two primary limitations: a heavy reliance on high-precision GPS/IMU data, which can fail in urban canyons or under dense foliage, and a dependence on specific environmental features like road markings or lamp posts. This research introduces a "scene-independent" registration method that relies on the inherent overlap between consecutive panoramic images rather than external landmarks or continuous hardware signals.
Methodology
The core of the proposed method is the application of the Relative Orientation Model (ROM) to spherical panoramic imagery. Instead of attempting to register LiDAR points to every image independently, the system calculates the relative movement between image pairs to propagate registration from a known starting point.
Relative Orientation Models
Two variations of the ROM are employed based on the available data:
- PA-ROM (Position/Attitude): Applied when both the camera's position and its orientation (tilt/rotation) are unknown.
- A-ROM (Attitude): Utilized when the camera's position is known (e.g., via GPS), but its orientation requires refinement.
Feature Matching and Refinement
To solve the ROM equations, the system automatically extracts feature points from panoramic image pairs using the SURF (Speeded Up Robust Features) algorithm. Because moving objects (like cars) or repetitive textures can create false matches, the RANSAC algorithm is combined with the ROM to filter out outliers, ensuring that only stable, static features are used for orientation calculations.
Experimental Evaluation
The method was evaluated using two road scene datasets captured with a Ladybug5 panoramic camera:
- Dataset I: A 250-meter road segment comprising 10 panoramic images and 3.8 million LiDAR points.
- Dataset II: A 350-meter segment comprising 7 panoramic images and 8.2 million LiDAR points.
Results and Discussion
The registration accuracy was assessed by manually selecting checkpoints between the LiDAR points and the panoramic images.
Accuracy and Stability
In Dataset I, the mean registration error was 7.36 pixels, while Dataset II achieved a superior accuracy of 3.75 pixels. The study also compared "Continuous Registration" (matching sequential frames like 1-2, 2-3) against "Discontinuous Registration" (matching non-sequential frames like 1-3, 1-4). The results indicated that continuous registration is significantly more stable, as increasing the distance between image pairs reduces the overlapping area and the number of reliable feature matches.
Error Accumulation
A key finding was the presence of "drift" over long sequences. In Dataset I, the registration error increased from 4.93 pixels at the start of the sequence to 10.67 pixels by the tenth image. This accumulation occurs because each frame's orientation is calculated relative to the previous one, allowing small errors to compound over time.
Conclusion
The ROM-based approach provides a flexible and accurate solution for MMS data registration that is less dependent on external hardware or specific road features. While the method demonstrates high precision in diverse environments, future work is needed to address error accumulation. Integrating global optimization techniques, such as loop closure commonly used in SLAM (Simultaneous Localization and Mapping), could mitigate drift and enable the registration of much longer image sequences.
Summary updated