On the Maximum Radius of Polynomial Lens Distortion

Matthew J. Leotta, David Russell, Andrew MatraiWACV 2022

TL;DR

This paper identifies a critical, often overlooked issue with common polynomial lens distortion models: they can "fold back" on themselves, causing points far outside the camera's field of view to project incorrectly into the image. The authors derive a closed-form solution to calculate the maximum radius for which these models remain bijective, providing a crucial tool for filtering invalid projections and validating lens models.

What Problem Does This Paper Address?

In computer vision, real-world lenses introduce distortions that deviate from the ideal pinhole camera model. Polynomial models are widely used to correct these distortions, but they possess a peculiar behavior: beyond a certain radius, the distortion function can become non-monotonic, effectively folding back on itself. This means points that should project far outside the image can erroneously appear within it. This "folding" can lead to significant errors in applications like augmented reality, where 3D scene geometry is projected and overlaid onto images. While this phenomenon is observed in practice and discussed in forums, a clear theoretical explanation and a method to quantify it have been lacking. This paper aims to fill that gap by providing a mathematical derivation and practical tools to detect and mitigate this issue.

Methods

The core of the paper lies in analyzing the mathematical properties of polynomial lens distortion models. The authors focus on the radial distortion function, $D(r)$, which maps an undistorted radius $r$ to a distorted radius $D(r)$. The "folding" occurs when $D(r)$ is no longer monotonic, meaning it's not a bijection. This happens when the derivative, $D'(r)$, becomes negative. The paper's key contribution is deriving the closed-form expression for the maximum valid radius, $r_{max}$, which is the smallest positive root of $D'(r) = 0$.

They tackle this by:

  • Deriving $D'(r)$ for common polynomial models: Specifically, they derive the derivative for the basic third-degree polynomial ($1 + k_1r + k_2r^2 + k_3r^3$) and the Brown-Conrady model ($1 + k_1r^2 + k_2r^4 + k_3r^6$).
  • Solving for the roots of $D'(r)$: They present solutions for first, second, and third-degree polynomials. For the third-degree case, they utilize the cubic formula, including a detailed breakdown for real and complex roots.
  • Developing an Algorithm (MinPosRoot): They provide a practical algorithm (Algorithm 1) to compute the smallest positive root of the general polynomial equation $1 + ar + br^2 + cr^3 = 0$, which is the core of finding $r_{max}$.
  • Applying to specific models: Algorithm 2 demonstrates how to use MinPosRoot for the Brown-Conrady model by transforming its coefficients.

Datasets

The paper uses the Lensfun database for evaluation. This is an open-source library and database containing over 5000 photographic lens models and their properties, widely used in production software. The authors specifically accessed radial distortion models from this database to assess the prevalence of the folding problem in real-world lenses. They filtered for "rectilinear" lens models.

Key Insights and Ideas

  • Polynomial models aren't always bijective: The most significant insight is that standard polynomial lens distortion models, despite their widespread use, can fail to be one-to-one mappings beyond a certain radius. This is a fundamental property that is often overlooked.
  • Folding can project "impossible" points: The folding phenomenon means that points far outside the camera's actual field of view can be mapped back into the image plane, leading to incorrect geometry and rendering artifacts.
  • $r_{max}$ is a critical parameter: The derived maximum valid radius ($r_{max}$) is not just a theoretical curiosity; it's a practical threshold. It defines the domain where the distortion model is well-behaved and bijective.
  • Negative infinity is the common problem: For models with a finite $r_{max}$, the distortion function often tends towards negative infinity. This means the distortion curve "turns back" and crosses itself, leading to the problematic projections.
  • Small coefficient changes matter: The paper highlights that even small perturbations in estimated polynomial coefficients can drastically change $r_{max}$, potentially shifting from an infinite radius (no folding) to a very small one (folding within the image).

Key Results

  • Prevalence of folding: Experiments on the Lensfun database reveal that 30.63% of lens models exhibit a finite $r_{max}$, meaning they have a folding problem.
  • Common issue with poly3: The "poly3" model type in Lensfun is particularly prone to this issue, with 47.13% of its instances having a finite $r_{max}$.
  • Negative infinity is the norm: For models with finite $r_{max}$, 98.26% tend towards negative infinity, indicating the problematic "folding back" behavior.
  • Most models are valid for image bounds: While many models fold, the majority (over 99%) have $r_{max}$ that lies outside the image bounds, meaning the folding doesn't manifest as artifacts within the typical image area. However, this still impacts applications projecting geometry.
  • Visualizations confirm theory: Figure 4 and Figure 6 visually demonstrate how different degree polynomials behave, showing clear folding artifacts for first and third-degree models but not for the second-degree case, aligning with the theoretical derivations.

Limitations and Future Work

  • Focus on radial distortion: The paper primarily addresses radial distortion and explicitly ignores tangential distortion, which is less common in practice but could still contribute to complex projection behaviors.
  • Polynomial models only: The analysis is limited to polynomial distortion models. While these are the most common, other distortion models exist and might have different failure modes.
  • Heuristic thresholds: The authors acknowledge that other heuristic thresholds can be used for filtering, but argue that $r_{max}$ provides a more principled approach tied to the model's inherent properties.
  • Future Work: The authors suggest exploring the use of $r_{max}$ as a constraint during the optimization of polynomial coefficients to ensure models are bijective or have $r_{max}$ greater than a target radius. This could lead to more robust lens model estimation.

Summary updated

BibTeX
@inproceedings{leotta2022maximum,
  author = {Matthew J. Leotta and David Russell and Andrew Matrai},
  title = {{On the Maximum Radius of Polynomial Lens Distortion}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2022},
}
Figure