Camera calibration is the mathematical bridge between the 2D pixel grid and the 3D physical world. Traditionally, determining a camera's intrinsic parameters—such as its focal length, principal point, or lens distortion—requires a specialized calibration target, like a checkerboard pattern with precisely known dimensions.
However, a powerful alternative paradigm has emerged: self-calibration (or autocalibration). Instead of relying on artificial markers, these techniques exploit fundamental geometric and physical invariants already embedded within the scene or the imaging process itself. The physical world obeys structural rules—such as orthogonality, isometry, and the physics of light reflection—that become distorted in predictable ways when viewed through an uncalibrated lens. By forcing the imaging model to respect these invariants, a system can deduce its own internal parameters directly from the data.
Projective Geometry and Orthogonality Constraints
How can a camera learn its own parameters simply by looking at an unstructured scene? The answer lies in the preservation of mathematical invariants under perspective projection.
A foundational insight comes from Bill Triggs in Autocalibration from Planar Scenes, who demonstrated that orthogonal direction vectors within a 3D planar scene must project to orthogonal directions in the camera's calibrated frame. In an uncalibrated image, perspective projection distorts these right angles. However, because the underlying 3D orthogonality is invariant to camera motion, any observed distortion provides a direct mathematical constraint on the Image of the Absolute Conic (IAC)—an imaginary geometric construct that encodes the camera's intrinsic matrix.
This concept of leveraging orthogonality manifests in various creative ways across the literature:
- Pedestrian Strides: Shiyao Huang et al. showed in Camera Calibration from Periodic Motion of a Pedestrian that the periodic, linear stride of a walking human defines three mutually orthogonal vanishing points in 3D space. In the image plane, these vanishing points form a triangle whose orthocenter corresponds exactly to the camera's principal point, allowing for calibration from ordinary surveillance footage.
- Concentric Circles: Under perspective projection, circles distort into ellipses, losing their apparent centers. However, Haifei Huang et al. demonstrated in The common self-polar triangle of concentric circles and its application to camera calibration that concentric circles share an infinite number of "common self-polar triangles." Crucially, these triangles are consistently right-angled in the metric frame. Enforcing this intrinsic right-angle property provides strong linear constraints on the IAC, pinning down camera intrinsics without needing a separate estimation of the vanishing line. Qing Wang 0006 later extended this principle to separate, non-concentric circles in Common self-polar triangle of separate circles for light field camera calibration, proving that the underlying projective geometry remains robust even without shared centers.
- Environmental Lines: In omnidirectional vision, Christopher Geyer et al. demonstrated in Catadioptric Camera Calibration that straight lines in the environment project to specific circular arcs on a parabolic mirror. By fitting these curves to internal scene features, the system can calibrate omnidirectional sensors even when the physical mirror boundary is occluded or misaligned.
Physics-Based Constraints from Reflections
While pure geometry relies on lines and shapes, physics-based self-calibration exploits the laws governing how light interacts with matter. Reflections from mirrors, glass, or water surfaces act as virtual secondary cameras, encoding the primary camera's intrinsics within the physics of the reflection itself.
In catadioptric stereo systems (cameras combined with mirrors), the reflection geometry provides strict constraints. Joshua Gluckman et al. showed in Planar Catadioptric Stereo: Geometry and Calibration that a camera's focal length can be recovered from a single image by analyzing the projection of the reflection's screw axis and horizon line. Because a mirror reflection creates a perfectly symmetrical virtual viewpoint, any internal camera distortion breaks this symmetry, allowing the system to self-calibrate.
This concept is pushed further when the geometry of the reflector itself is unknown. Kai Han et al. demonstrated in Mirror Surface Reconstruction under an Uncalibrated Camera that observing the reflections of a moving reference screen across multiple unknown positions provides enough geometric consistency to simultaneously solve for both the camera's intrinsics and the 3D shape of the mirror. Similarly, Kosuke Takahashi et al. showed in A Linear Extrinsic Calibration of Kaleidoscopic Imaging System from Single 3D Point that the multiple virtual points created by a single 3D point reflecting inside a kaleidoscope must obey strict planar reflection symmetries, providing enough constraints to linearly solve for mirror orientations. Even the external boundaries of an object can drive calibration; Keith Forbes et al. demonstrated in Shape-from-Silhouette with Two Mirrors and an Uncalibrated Camera that silhouettes generated by mirror reflections must form intersecting visual cones that consistently bound the same 3D object, yielding enough information for full camera self-calibration.
[Real Object] ------> Mirror 1 ------> [Virtual View 1] \
| +--> [Uncalibrated Camera]
v | (Inconsistencies reveal
Mirror 2 -------------------------> [Virtual View 2] / calibration errors)
Even everyday surfaces like glass plates or water bodies provide calibration cues via the Fresnel equations, which dictate that the intensity of reflected light varies based on the incident angle. Qian Zheng et al. in What Does Plate Glass Reveal About Camera Calibration? and Ryo Kawahara et al. in Appearance and Shape from Water Reflection utilized this angle-dependent radiance variation. By minimizing the radiance error between the direct view of a scene and its reflected counterpart, their algorithms solve for the camera's focal length. The physics of light reflection acts as an implicit geometric sensor: if the assumed focal length is incorrect, the calculated incident angles will fail to satisfy the Fresnel equations, exposing the calibration error.
Kinematic and Material Invariants
Beyond static geometry and reflections, self-calibration can leverage the physical properties of moving objects or the consistency of light transport over time.
Consider a deforming object captured by a monocular camera. In Incremental Non-Rigid Structure-from-Motion with Unknown Focal Length, Thomas Probst et al. leveraged the physical constraint of isometry—the principle that the distance between points on a surface (like cloth or bending paper) remains constant during deformation.
Perspective projection scales an object's apparent size inversely with depth, a process governed directly by the focal length ($f$). If an optimization algorithm assumes an incorrect focal length, a 3D object undergoing pure bending will appear to unnaturally stretch or compress in 3D space to explain its 2D video projection. Enforcing strict physical isometry penalizes these non-rigid artifacts, forcing the system to converge on the true focal length.
Incorrect Focal Length (f) Guess:
2D Motion Frame 1 -> [ Reconstructed 3D Shape ] (Normal)
2D Motion Frame 2 -> [ Reconstructed 3D Shape ] (Unnaturally Stretched) -> Isometry Violated!
Correct Focal Length (f) Guess:
2D Motion Frame 1 -> [ Reconstructed 3D Shape ] (Normal)
2D Motion Frame 2 -> [ Reconstructed 3D Shape ] (Normal) -> Isometry Satisfied!
Similarly, radiometric and structural priors can reduce the complexity of the calibration problem:
- Radiometric Consistency: Wiennat Mongkulmann et al. demonstrated in Auto-Radiometric Calibration in Photometric Stereo that the physical consistency between surface normals and light reflection allows a system to simultaneously recover a camera's non-linear response function and the object's 3D shape without specialized hardware.
- Active Illumination: In active systems, Yotam Erel et al. showed in Neural Projection Mapping Using Reflectance Fields that projecting randomized color patterns breaks geometric symmetry. This provides unique visual anchors that allow a differentiable pipeline to jointly backpropagate and recover a projector's spatial pose and internal optics.
- Structural Invariants: Sometimes, simply knowing a single hardware invariant can collapse the dimensionality of the problem. José I. Ronda et al. showed in Autocalibration with the Minimum Number of Cameras with Known Pixel Shape that assuming a known pixel shape (such as square pixels) eliminates enough degrees of freedom to reduce a complex three-dimensional search for the plane at infinity into an efficient two-dimensional optimization, enabling autocalibration from as few as five images.
Key Foundational Work
Autocalibration from Planar Scenes (1998) — contributed 2 concept elements to this direction.
Planar Catadioptric Stereo: Geometry and Calibration (1999) — contributed 1 concept elements to this direction.
Catadioptric Camera Calibration (1999) — contributed 1 concept elements to this direction.