Camera Self Calibration with Varying Intrinsics

45 concepts45 papers

In classical computer vision, a camera is often treated as a rigid, unchanging instrument. When algorithms estimate 3D structure from a sequence of images, they typically rely on the constant intrinsic assumption. This assumption states that the camera's internal geometry—such as its focal length, principal point, and pixel aspect ratio—remains completely static across all captured frames.

Under this assumption, the internal calibration matrix $K$ is constant. This allows geometric frameworks to elegantly isolate camera motion (rotation and translation) from the scene's structure.

However, real-world videography routinely violates this constraint. Whether it is a broadcast sports camera zooming in on a play, a smartphone camera dynamically adjusting its focus, or a drone sensor experiencing structural drift due to temperature fluctuations, camera optics are inherently dynamic. When the internal parameters of a camera change from frame to frame, traditional geometric equations break down, leading to catastrophic failures in 3D reconstruction, depth estimation, and object tracking.

The research direction of dynamic camera calibration addresses this challenge by discarding the static assumption and modeling camera intrinsics as time-varying parameters that must be estimated on the fly.

The Geometry of Changing Optics

To understand why variable intrinsics disrupt traditional computer vision, consider the visual effect of a simple zoom. When a camera increases its focal length, the field of view narrows, and the features in the image expand outward from the center.

To a geometric solver that assumes a fixed focal length, this uniform expansion is mathematically indistinguishable from the camera physically moving forward along its optical axis. This creates a fundamental ambiguity between intrinsic scaling and extrinsic translation. As shown by Peizhe Jiang et al. in Detaching and Boosting: Dual Engine for Scale-Invariant Self-Supervised Monocular Depth Estimation, variations in focal length mimic physical motion cues, causing standard self-supervised models to produce highly inconsistent and erroneous depth predictions.

Static Intrinsics (Standard SfM):
Frame 1 (K) --------> Frame 2 (K) --------> Frame 3 (K)   [Only R and t change]

Dynamic Intrinsics (Real World):
Frame 1 (K₁) -------> Frame 2 (K₂) -------> Frame 3 (K₃)   [Both K, R, and t change]

Mathematically, self-calibration algorithms operate by tracking how the projection of a theoretical geometric entity—the Absolute Conic—behaves under perspective projection. If the intrinsics are constant, the image of the absolute conic remains invariant across all views, providing a rigid set of constraints (such as the Kruppa equations or the Dual Absolute Quadric) to solve for calibration.

When the focal length changes dynamically, the absolute conic's image changes in every single frame. In a seminal paper on the classification of ambiguities, Peter F. Sturm demonstrated in Critical Motion Sequences for Monocular Self-Calibration and Uncalibrated Euclidean Reconstruction that ignoring these variations excludes common filming techniques, like zooming, from the mathematical foundations of self-calibration.

When these constraints are violated, classical formulations fail. For instance, Yi Ma et al. in Kruppa Equation Revisited: Its Renormalization and Degeneracy and Adlane Habed et al. in Efficient Pruning LMI Conditions for Branch-and-Prune Rank and Chirality-Constrained Estimation of the Dual Absolute Quadric note that traditional absolute quadric and algebraic solvers become highly unstable or completely degenerate if they cannot explicitly account for a changing calibration matrix.

Downstream Failures in Dynamic Environments

The breakdown of geometric consistency under variable optics cascades into high-level computer vision tasks. When downstream models assume a fixed projection matrix, any unmodeled change in focal length distorts their spatial understanding:

Formulating Dynamic Calibration Frameworks

To resolve the scale-motion ambiguity, modern research focuses on formulating algorithms that treat camera intrinsics as time-varying variables within the optimization loop.

Analytical and Minimal Solvers

One approach expands classical geometric solvers to include focal length as an explicit per-frame or per-view variable. Peter F. Sturm et al. pioneered this in On Plane-Based Camera Calibration: A General Algorithm, Singularities, Applications by introducing a linear calibration algorithm capable of handling an arbitrary number of views with changing focal lengths.

More recently, researchers have sought to find the mathematical minimum number of point correspondences required to solve these systems. Andrea Porfiri Dal Cin et al. addressed this in Minimal Perspective Autocalibration, developing a comprehensive taxonomy of minimal problems that jointly estimate 3D point depths and time-varying camera intrinsics directly, bypassing the need for rigid fundamental matrices.

Online and Continuous Estimation

Instead of solving for calibration in isolated batches, online methods continuously track parameter drift. This is crucial because, as noted by Yiming Qian et al. in A Reliable Online Method for Joint Estimation of Focal Length and Camera Rotation and Jinjiang You et al. in Multi-Cali Anything: Dense Feature Multi-Frame Structure-from-Motion for Large-Scale Camera Array Calibration, camera parameters drift over time due to environmental factors like temperature fluctuations and physical handling.

By treating intrinsics as adjustable, time-varying parameters during optimization, these frameworks absorb the physical volatility of the lens. For example, Guohang Yan et al. demonstrated in Joint Camera Intrinsic and LiDAR-Camera Extrinsic Calibration that treating intrinsics as continuous variables within a joint optimization loop prevents calibration errors from propagating and ruining multi-sensor alignment.

Hardware-Assisted Constraints

When purely mathematical estimation proves too unstable, researchers leverage specialized hardware configurations to anchor the geometry. Emanuel Aldea et al. explored this in Hybrid Focal Stereo Networks for Pattern Analysis in Homogeneous Scenes.

Instead of relying on a single camera that changes its focal length—which introduces mathematical instability because distortion and zoom must be re-estimated simultaneously—the authors proposed a hybrid stereo system. By pairing a static wide-angle camera (which maintains a stable global calibration) with a dynamic zoom camera (which captures high-resolution details), the system anchors the global geometric structure while allowing the zoom parameters to vary safely.

Key Foundational Work

An Optimal Solution for Mobile Camera Calibration (1990) — contributed 2 concept elements to this direction.
Maintaining stereo calibration by tracking image points (1993) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_dynamic_camera_calibration_71fd024c,
  title = {dynamic camera calibration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-camera-calibration/rdr_71fd024c21ba47e99f534ac4d0fa2399},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_dynamic_camera_calibration_71fd024c_v1,
  title = {dynamic camera calibration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/dynamic-camera-calibration/rdr_71fd024c21ba47e99f534ac4d0fa2399/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}

See something wrong? Report a mistake