When we reconstruct a 3D scene or calibrate a camera from a sequence of images without a known target (a process known as autocalibration), we are solving a geometric puzzle. Traditionally, this is framed as an algebraic optimization problem: find a set of camera matrices and 3D points that minimize projection errors.
However, unconstrained algebraic optimization suffers from a fundamental flaw: mathematics is far more permissive than physics. A solver operating purely in an abstract projective space can easily find a mathematically optimal solution that corresponds to a physical impossibility—such as an imaginary focal length or a camera matrix that cannot exist in the real world. As Porfiri Dal Cin et al. observe in Minimal Perspective Autocalibration, estimating intermediate mathematical proxies without enforcing physical plausibility frequently yields invalid optical configurations.
The Failure of Post-Hoc Corrections
Why not just optimize first and fix the results later? Historically, algorithms minimized algebraic errors and then applied post-hoc corrections to force the parameters into a valid physical form. But this separation is highly fragile.
As Kocur et al. demonstrate in Robust Self-Calibration of Focal Lengths from the Fundamental Matrix, even minute errors in an estimated fundamental matrix can cause classical algebraic formulas to collapse, yielding non-physical results like imaginary focal lengths. When physical constraints are omitted from the optimization loop, solvers are easily pulled into mathematically valid but physically impossible local minima.
The authors of Globally Optimal Algorithms for Stratified Autocalibration argue that post-hoc corrections are inherently unstable. Instead, constraints like chirality (ensuring points lie in front of the camera) and matrix validity must be enforced directly within the optimization loop to prevent the solver from wandering into impossible regions of the search space in the first place.
The Absolute Conic and Positive Semidefiniteness
To understand how to enforce these constraints natively, we must look at the geometry of camera intrinsics. A central concept in autocalibration is the Dual Image of the Absolute Conic (DIAC). The DIAC is a matrix representation that encodes a camera's internal parameters, such as its focal length and principal point. Crucially, for a DIAC to represent a real, physical camera, it must be positive semidefinite.
Early foundations by Triggs in Autocalibration and the absolute quadric showed that enforcing physical rank constraints through non-linear optimization significantly bolsters numerical stability against noise. To guarantee that the estimated DIAC is always physically meaningful, researchers turned to Semidefinite Programming (SDP)—a subfield of convex optimization perfectly suited for handling positive semidefinite matrix constraints.
In Self-Calibrating Cameras Using Semidefinite Programming, Shen et al. reformulate autocalibration by treating positive-definiteness not as a final check, but as a hard constraint within an SDP solver. Because the formulation is convex, it guarantees convergence to a global optimum while strictly avoiding physically impossible solutions. Similarly, Agrawal et al. apply SDP to multi-camera calibration in Camera calibration using spheres: A semi-definite programming approach, demonstrating that explicitly constraining the DIAC to be positive definite prevents optimization from degrading under high noise levels.
┌────────────────────────────────────────────────────────┐
│ Unconstrained Optimization │
│ Solves for mathematical proxies (e.g., Least Squares) │
└───────────────────────────┬────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────┐
│ Mathematically Optimal but Physically Impossible │
│ (e.g., Imaginary Focal Lengths, Negative DIAC) │
└────────────────────────────────────────────────────────┘
VS.
┌────────────────────────────────────────────────────────┐
│ Semidefinite Programming (SDP) │
│ Bakes positive-semidefinite constraints into solver │
└───────────────────────────┬────────────────────────────┘
│
▼
┌────────────────────────────────────────────────────────┐
│ Globally Optimal & Physically Consistent │
│ (Guaranteed valid optical configurations) │
└────────────────────────────────────────────────────────┘
Grounding Optimization in Physical Parameters
Beyond SDP, another way to ensure physical validity is to parameterize the optimization space using real physical variables rather than abstract projective matrices.
The authors of A Simple Technique for Self-Calibration show that directly searching for physical camera parameters like focal length provides superior numerical stability and convergence compared to searching through abstract projective spaces. This philosophy is echoed in image rectification: Cheng et al. demonstrate in Self Image Rectification for Uncalibrated Stereo Video with Varying Camera Motions and Zooming Effects that parameterizing homographies directly through focal lengths and 3D rotations prevents the extreme, unnatural warping typical of unconstrained mathematical transformations.
A Broader Paradigm for Geometric Rigor
The insight of embedding physical laws directly into optimization extends across computer vision. Whether recovering 3D structures, camera response curves, or matrix factorizations, unconstrained optimization consistently fails where prior-constrained optimization succeeds.
- Matrix Factorization: In Large-Scale Matrix Factorization with Missing Data under Additional Constraints, Mitra et al. introduce a low-rank semidefinite programming framework to bake physical constraints like orthonormality directly into matrix factorization, ensuring reconstructed motion tracks align with physical realities like circular rotation.
- Non-Rigid Motion: In the domain of non-rigid structure from motion, Iglesias et al. highlight in Accurate Optimization of Weighted Nuclear Norm for Non-Rigid Structure from Motion that future accuracy gains rely on integrating domain-specific physical constraints—such as joint limits or skeletal connectivity—directly into the factorized components of a model.
- Radiometric Calibration: Even when calibrating camera sensors rather than geometry, the principle holds. Mongkulmann et al. show in Auto-Radiometric Calibration in Photometric Stereo that empirical priors on camera response curves are vital to prevent solvers from finding mathematically valid but physically impossible sensor behaviors.
By shifting from unconstrained algebraic minimization to physically bounded convex optimization, these techniques transform geometric estimation from a fragile mathematical exercise into a robust, physically grounded reconstruction process.
Key Foundational Work
Autocalibration and the absolute quadric (1997) — contributed 1 concept elements to this direction.
A Simple Technique for Self-Calibration (1999) — contributed 1 concept elements to this direction.