Pose Estimation for Omni-directional Cameras using Sinusoid Fitting

Haofei Kuang, Qingwen Xu, Xiaoling Long, Sören Schwertfeger

Abstract: We propose a novel pose estimation method for geometric vision of omni-directional cameras. On the basis of the regularity of the pixel movement after camera pose changes, we formulate and prove the sinusoidal relationship between pixels movement and camera motion. We use the improved Fourier-Mellin invariant (iFMI) algorithm to find the motion of pixels, which was shown to be more accurate and robust than the feature-based methods. While iFMI works only on pin-hole model images and estimates 4 parameters (x, y, yaw, scaling), our method works on panoramic images and estimates the full 6 DoF 3D transform, up to an unknown scale factor. For that we fit the motion of the pixels in the panoramic images, as determined by iFMI, to two sinusoidal functions. The offsets, amplitudes and phase-shifts of the two functions then represent the 3D rotation and translation of the camera between the two images. We perform experiments for 3D rotation, which show that our algorithm outperforms the feature-based methods in accuracy and robustness. We leave the more complex 3D translation experiments for future work.

Introduction

Omni-directional cameras provide a 360-degree field of view, which is highly advantageous for autonomous robot navigation and visual odometry. However, these cameras often suffer from low resolution and significant distortion when projected into standard image formats. Traditional feature-based methods, such as ORB-SLAM, rely on identifying and tracking distinct landmarks like corners or edges. In environments with repetitive textures (e.g., grass) or low resolution, these methods frequently fail. This paper proposes a novel approach that treats pixel displacement across a panoramic image as a periodic signal, modeling it with sinusoidal functions to achieve robust 6-DOF pose estimation.

Methodology

The proposed method shifts away from local feature matching toward a global motion estimation framework. The pipeline consists of four primary stages:

  1. Unwrapping: The raw catadioptric (mirror-based) image is transformed into a cylindrical panorama. This projection is essential for the subsequent geometric modeling.
  2. Local Motion Estimation via iFMI: Instead of matching points, the system employs the improved Fourier-Mellin Invariant (iFMI) algorithm. Operating in the frequency domain, iFMI estimates x-y translation and rotation between image windows, providing robustness against noise and blur.
  3. The Sinusoidal Model: The core theoretical contribution is the proof that camera motion results in pixel shifts that follow a sinusoidal pattern in a cylindrical panorama: $y = B + A \sin(\omega x + \phi)$. Specifically, vertical shifts ($\Delta v$) are used to derive roll and pitch, while horizontal shifts ($\Delta u$) inform yaw and translation.
  4. Optimization: To handle noisy data from the iFMI step, the Levenberg-Marquardt algorithm with a Pseudo-Huber loss function is used to fit the data to the sinusoidal curves, effectively filtering out outliers.
Geometric model for unwrapping omni-directional images into panoramas.

Experimental Setup

The system was validated using a mobile robot platform equipped with a smartphone (OnePlus 5) and a Kogeto Dot omni-lens.

The experimental robot platform equipped with an omni-directional camera lens.

Testing was conducted in three distinct environments:

  • Grass: A challenging, feature-poor environment with repetitive textures.
  • Office: A feature-rich indoor setting.
  • Street: A typical outdoor scene with varied depth.

Ground truth for rotation was obtained via the smartphone's internal IMU.

Results and Discussion

The sinusoid fitting method was compared against standard feature-based trackers, ORB and AKAZE.

  • Accuracy: The proposed method achieved a total average error of 0.074 rad, significantly outperforming AKAZE (0.177 rad) and ORB (0.300 rad).
  • Robustness: In the "Grass" dataset, where standard trackers often lose lock, the sinusoidal model provided stable roll estimation by leveraging the global geometric constraint of the wave pattern. This "Motion as a Wave" insight allows the algorithm to ignore local errors like moving objects or blurry patches.
  • Efficiency: With a processing time of 0.40s per frame, the method is slower than ORB (0.11s) but nearly twice as fast as AKAZE (0.75s), offering a favorable trade-off between speed and precision for real-time applications.

Conclusion and Future Work

By modeling pixel flow as a sinusoidal wave, this research provides a robust alternative to feature-based visual odometry for omni-directional cameras. The global constraint of the sinusoid effectively regularizes motion estimation in ambiguous environments. Future work will focus on refining translation estimation—currently limited by assumptions of uniform scene depth—and integrating the method into a complete SLAM framework.

Summary updated

Abstract

We propose a novel pose estimation method for geometric vision of omni-directional cameras. On the basis of the regularity of the pixel movement after camera pose changes, we formulate and prove the sinusoidal relationship between pixels movement and camera motion. We use the improved Fourier-Mellin invariant (iFMI) algorithm to find the motion of pixels, which was shown to be more accurate and robust than the feature-based methods. While iFMI works only on pin-hole model images and estimates 4 parameters (x, y, yaw, scaling), our method works on panoramic images and estimates the full 6 DoF 3D transform, up to an unknown scale factor. For that we fit the motion of the pixels in the panoramic images, as determined by iFMI, to two sinusoidal functions. The offsets, amplitudes and phase-shifts of the two functions then represent the 3D rotation and translation of the camera between the two images. We perform experiments for 3D rotation, which show that our algorithm outperforms the feature-based methods in accuracy and robustness. We leave the more complex 3D translation experiments for future work.
BibTeX
@inproceedings{kuang2019pose,
  author = {Haofei Kuang and Qingwen Xu and Xiaoling Long and Sören Schwertfeger},
  title = {{Pose Estimation for Omni-directional Cameras using Sinusoid Fitting}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2019},
  url = {https://openreview.net/forum?id=s92jriyz0g},
}
Figure