TL;DR
This paper tackles the challenging problem of calibrating a very general class of cameras, called "central cameras," without needing any prior knowledge of the scene or calibration targets. They achieve this by cleverly using sequences of pure camera rotations and translations to derive geometric constraints on the camera's projection rays, enabling self-calibration.
What Problem Does This Paper Address?
Camera calibration is crucial for many computer vision tasks, allowing us to understand how 3D world points are projected onto a 2D image. Traditional methods often rely on specific camera models (like the pinhole camera) and require controlled environments with calibration grids or precisely known camera movements. This paper addresses the need for more flexible calibration methods that can handle a wider range of cameras, including those with complex distortions (like fisheye lenses), without requiring external calibration targets. The core challenge is to infer the camera's projection rays—the lines in 3D space that correspond to each pixel—using only image correspondences obtained from natural camera motion. They focus on "central cameras," which have a single effective viewpoint, simplifying the problem while still encompassing many real-world cameras.
Methods
The paper's core contribution lies in deriving geometric constraints from specific types of camera motion: pure translations and pure rotations.
- Flowcurves: They introduce the concept of "flowcurves," which are sets of matching pixels across a sequence of images. For a pure translation, a flowcurve's corresponding projection rays are coplanar. For a pure rotation, these rays lie on a cone. These geometric properties are the foundation for their self-calibration algorithms.
- Translational Motion: Using only pure translations, they show that self-calibration is possible, but only up to an affine transformation of the projection rays. This means the relative directions of rays are known, but their absolute orientation in space is not fully determined. They propose an algorithm using at least four translational motions in different directions to achieve this affine calibration.
- Rotational Motion: Pure rotations provide stronger constraints. They parameterize projection rays based on the rotation axis and an opening angle for the cone. They also introduce assumptions about the rotation axis piercing the image and the ability to determine the rotation angle from closed rotational flowcurves.
- Combined Motions: The main contribution is in combining these motions for full metric self-calibration:
- One Rotation + One Translation: This combination allows for self-calibration up to a single degree of freedom, analogous to an unknown focal length in a pinhole camera.
- Two Rotations + One Translation: Adding a second rotational motion allows them to resolve the ambiguity and achieve full metric self-calibration.
- Two Rotations: This scenario also enables full metric self-calibration by carefully analyzing the intersection of flowcurves from different rotations.
Datasets
Not discussed in the paper. The authors mention using simulated data and real fisheye camera images. For real-world experiments, they used a Nikon coolpix fisheye lens (FC-E8) and performed controlled translations and rotations.
Key Insights and Ideas
- Flowcurves as Geometric Priors: The concept of flowcurves is a powerful way to extract geometric information from image sequences without explicit scene knowledge. The coplanarity (translation) and conicity (rotation) of projection rays derived from flowcurves are key.
- Affine vs. Metric Calibration: The paper clearly distinguishes between affine calibration (relative ray directions) achievable with translations alone, and metric calibration (absolute ray directions) requiring rotations. This highlights the fundamental limitations of translation-only approaches.
- Flexibility of Generic Models: By focusing on a generic central camera model, the methods are applicable to a wide range of cameras, including those with significant distortions, which is a significant step beyond traditional pinhole camera calibration.
- Minimal Motion Requirements: The paper explores the minimal sets of motions (e.g., two rotations) required for full self-calibration, providing theoretical insights into the problem's complexity.
Key Results
- Self-calibration up to an affine transformation is achievable with at least four translational motions.
- Self-calibration up to one degree of freedom (like unknown focal length) is possible with one rotation and one translation.
- Full metric self-calibration is achievable with two rotations and one translation, or with two rotations.
- Experimental results on simulated and real fisheye images demonstrate the effectiveness of their methods in removing significant distortion.
Limitations and Future Work
- Assumptions: The algorithms rely on several assumptions, including the rotation axis piercing the image, the ability to determine rotation angles from closed flowcurves, and the existence of dense matches. These assumptions might not hold for all camera setups or motions.
- Flowcurve Determination: The accuracy of the self-calibration heavily depends on the quality of the flowcurves (dense matches). The paper mentions using simple interpolation and fitting conics, suggesting that more sophisticated methods for flowcurve extraction and handling noise would improve robustness.
- General Motions: The current work focuses on pure translations and rotations. Future work aims to extend these approaches to more general camera motions.
- Full Image Plane Calibration: The current methods might only calibrate a restricted region of the image plane. Extending this to the entire image plane for all camera types is a goal.
- Noise Robustness: While simulations show reasonable performance with noise, practical implementations will require more robust techniques to handle real-world noise in image matches.
Summary updated