Polarization calibration of fisheye cameras using skylight in the wild

Shuo Wang, Shiqi Che, Dianze Dong, Wenxin LiuarXiv 2026 · arXiv

TL;DR

This paper introduces a "calibration in the wild" method for fisheye polarization cameras that uses natural skylight instead of expensive laboratory equipment. By jointly modeling lens-induced distortions and sensor imperfections in a unified eight-parameter framework, the researchers achieved sub-degree accuracy across the entire field of view, significantly improving the reliability of polarization-based robotic navigation.

What Problem Does This Paper Address?

For robots to navigate autonomously, they need a reliable "compass." Skylight polarization—the pattern of light scattering in the atmosphere—provides a natural directional cue that many insects use. Fisheye lenses are ideal for capturing this pattern because of their wide field of view.

However, there’s a catch: fisheye lenses and the polarization sensors (analyzers) behind them aren't perfect. The lens itself changes the polarization of light as it passes through, especially at steep angles near the edges of the image. Standard calibration methods usually only fix the sensor's internal errors, ignoring these significant lens effects. Until now, correcting the full field of view required complex, specialized lab setups that are difficult for most researchers to access or deploy in the field.

Methods

The authors propose a joint lens–analyzer calibration framework that models the entire optical path in one go.

  • Unified 8-Parameter Model: The core is a model that accounts for two lens parameters (modeling incidence-dependent transmission) and six analyzer parameters (gains and angular offsets for the sensor's micro-polarizers).
  • "In the Wild" Data Collection: Instead of a lab, the method uses a short (approx. 10-second) handheld video of the sky. As the user rotates the camera, different parts of the sky pattern are seen by different parts of the lens.
  • Consistency-Based Optimization: The system doesn't need to know the "true" polarization of the sky. Instead, it uses cross-frame consistency: if the camera rotates, the same point in the sky should still yield the same polarization measurement after correction.
  • Geometric Alignment: They use standard visual features (SIFT) and RANSAC on the total intensity images to precisely track the camera's rotation, which then serves as the ground truth for the polarization consistency check.
Fig. 2. Overview of the proposed sky-based joint calibration framework. (a) Sky observations are filtered and paired using fixed geometric correspondences. (b) A shared eight-parameter lens–analyzer model corrects both observations. (c) Robust joint optimization combines cross-frame polarization consistency, four-channel consistency, and a lens-response penalty. (d) The estimated lens and analyzer parameters enable subsequent Stokes, AoLP, and DoLP correction.

Datasets

The researchers evaluated their method on real-world outdoor sequences:

  • Data A: A 9.5-second handheld sequence (142 images) used to estimate the 8 parameters.
  • Data B: A 10.6-second independent sequence (159 images) captured at a different time under cloudier skies to test how well the calibration "transfers" without refitting.
  • Robot Validation: They also mounted the camera on a Unitree Go2 quadruped robot to validate solar-azimuth estimation in a practical navigation scenario.

Key Insights and Ideas

  • The Lens is a Major Culprit: The paper demonstrates that lens-induced distortion is a dominant error source in peripheral regions. Ignoring it leads to massive errors at high incidence angles.
  • Symmetry Simplifies Everything: By assuming the lens is axis-symmetric, they can model its complex 3D effect using a simple 1D polynomial based on the incidence angle $\theta$.
  • Redundancy is a Feature: Most polarization cameras (DoFP) have four channels but only need three to define linear polarization. The authors exploit this 4th-channel redundancy as a "self-check" constraint during optimization.
  • Transferability: A key "aha" moment was finding that parameters learned on a clear day (Data A) still worked perfectly on a cloudy day (Data B), proving the model captures the hardware's intrinsic properties rather than just over-fitting to a specific sky state.

Key Results

The quantitative improvements are substantial:

  • Sub-Degree Accuracy: The method achieved a mean absolute error (MAE) for the Angle of Linear Polarization (AoLP) of less than 0.5° across the entire field of view.
  • Edge Performance: In the critical peripheral regions (80°–90° incidence), the error was reduced by 73.4% compared to uncalibrated data.
  • Solar Alignment: In robot experiments, the estimated solar azimuth (the sun's horizontal direction) aligned much more closely with the visible sun after calibration.
Fig. 1. Axisymmetric lens response modeled in a single plane of incidence. (a) Fisheye camera geometry and incidence angle θ. (b) Blue-channel Mueller matrix parameter B(θ).

Limitations and Future Work

While highly effective, the authors note a few areas for growth:

  • Sky Conditions: The method relies on having enough "valid" sky pixels. Extremely heavy cloud cover or total occlusion might make the initial "in the wild" calibration difficult.
  • Spectral Range: The current study focused on the blue channel, which is standard for skylight navigation. Future work could extend this to full RGB or multi-spectral polarization.
  • System Integration: The next step is integrating this calibrated "celestial compass" into a full visual-inertial navigation system (VINS) to see how much it reduces long-term heading drift in GPS-denied environments.

The authors plan to release their calibration software as an open-source tool, making high-accuracy wide-field polarization sensing accessible to the broader robotics community.

Summary updated

Abstract

Skylight polarization provides directional cues for robotic navigation, but fisheye lens effects and analyzer imperfections can degrade polarization measurements across the field of view. We present a joint lens-analyzer calibration method that explicitly models incidence-dependent lens transmission together with analyzer gains and angular offsets in a unified eight-parameter formulation. Given known geometric calibration, rotations between camera frames can be estimated using visual features and cross-frame directional polarized light consistency can be used for polarization calibration. Therefore, our calibration approach requires only a short handheld outdoor sequence under natural skylight, without dedicated polarization calibration equipment or a predefined sky polarization model. On real-world outdoor sequences, the proposed method achieves a weighted cross-frame angle-of-linear-polarization (AoLP) mean absolute error below 0.5° across all incidence angles, with a 73.4% reduction in the 80°-90° incidence angle region, and generalizes to an independent sequence with an overall mean absolute error of 0.466° without refitting. Preliminary solar-azimuth experiments using a camera mounted on a quadruped robot further show improved alignment with the visible solar center, supporting the potential of calibrated AoLP measurements for relative-heading estimation and future polarization--inertial navigation. The calibration software tool will be released upon acceptance of this paper.
BibTeX
@misc{wang2026polarization,
  author = {Shuo Wang and Shiqi Che and Dianze Dong and Wenxin Liu},
  title = {{Polarization calibration of fisheye cameras using skylight in the wild}},
  year = {2026},
  eprint = {2611.08911},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.08911},
}
Figure