ArUcOmni: detection of highly reliable fiducial markers in panoramic images

Jaouad Hajjami, Jordan Caracotte, Guillaume Caron, Thibault NapoléonWorkshops @ CVPR 2020 · DOI

Abstract

ArUcOmni is a novel adaptation of the ArUco fiducial marker system designed specifically for panoramic and fisheye cameras. By replacing standard planar homographies with a unified spherical projection model, the system achieves near-perfect marker detection and significantly more accurate 3D pose estimation in images with extreme wide-angle distortion.

Introduction

Fiducial markers, such as the black-and-white square ArUco tags, are essential for robot navigation and augmented reality, providing reliable landmarks for camera localization. However, traditional systems rely on the "pinhole camera" assumption, where straight lines in the physical world remain straight in the image. Panoramic sensors, including catadioptric mirrors and fisheye lenses, offer a vast field of view but introduce non-linear distortions that curve these lines, causing standard detection algorithms to fail.

Detection of markers in a highly distorted panoramic image using the ArUcOmni system.

Existing solutions often involve image rectification, which can crop the field of view or introduce resolution artifacts. ArUcOmni addresses this by detecting markers across the entire distorted image without loss of information.

Methodology

The primary innovation of ArUcOmni is the transition from a 2D planar detection pipeline to a 3D spherical model.

  1. Unified Spherical Model: Pixels are projected onto a unit sphere, a model capable of representing both catadioptric and fisheye sensors.
  2. Spherical Homography: Instead of standard 2D transformations, ArUcOmni calculates transformations through the sphere. Detected corners are projected onto the sphere and then "unwarped" onto a virtual camera plane.
  3. Warping and Extraction: This spherical approach allows the system to extract clean, undistorted binary codes from markers, even those located at the extreme edges of the field of view.
Extracted and undistorted binary codes from markers located in high-distortion zones.
  1. Refined Pose Estimation: The 3D pose is calculated using a specialized 2D-3D correspondence method for spherical coordinates, followed by non-linear optimization using the Levenberg-Marquardt algorithm to minimize reprojection error.

Experimental Evaluation

The authors introduced the ArUcOmni dataset, consisting of 225 high-resolution images from catadioptric and fisheye cameras. A 3D marker rig—a cube with markers on three orthogonal faces—was used to establish ground truth for pose estimation accuracy.

Key Results

ArUcOmni was compared against standard ArUco and a rectified version of ArUco:

  • Detection Rate: In catadioptric images, ArUcOmni achieved a detection rate of nearly 100%, whereas standard and rectified versions often dropped below 10% for peripheral markers.
  • Pose Accuracy: ArUcOmni significantly outperformed competitors. In catadioptric tests, it achieved an orientation error of 1.39°, compared to 3.93° for the rectified method and 19.2° for standard ArUco.
  • Fisheye Performance: While generally superior, ArUcOmni faced challenges at extreme 90-degree angles where spherical polygon approximation becomes difficult, occasionally allowing the rectified method to perform better in specific edge cases.

Conclusion and Future Work

ArUcOmni demonstrates that performing mathematical operations on a spherical representation preserves image data and resolution across the entire field of view. However, the system's initial polygon extraction still relies on finding straight lines in binarized images, which are curved in panoramic views. Future work will focus on adapting the entire pipeline, including initial contour detection, to work directly on spherical representations to improve robustness at extreme angles.

Summary updated

BibTeX
@inproceedings{hajjami2020arucomni,
  author = {Jaouad Hajjami and Jordan Caracotte and Guillaume Caron and Thibault Napoléon},
  title = {{ArUcOmni: detection of highly reliable fiducial markers in panoramic images}},
  booktitle = {Workshops @ CVPR},
  year = {2020},
  doi = {10.1109/cvprw50498.2020.00325},
}
Figure