In computer vision, reconstructing a 3D scene from 2D images is often framed as finding the roots of a polynomial system. Whether we are estimating the relative pose of two cameras or the focal length of a lens, the underlying mathematics—often referred to as Algebraic Vision—is elegant but indifferent to reality. A polynomial solver might return several mathematically valid solutions, but only one corresponds to the physical world. The rest are "spurious" or "ghost" solutions: configurations where points exist behind the camera, focal lengths are negative, or objects are mirrored across impossible planes.
The research direction across these papers focuses on bridging this gap by using physical constraints to disambiguate, filter, and accelerate geometric reconstruction.
The Gap Between Algebra and Physics
The fundamental problem, as explored by Joe Kileel and Kathlén Kohn in Snapshot of Algebraic Vision, is that standard algebraic solvers treat projections as infinite lines. In reality, a camera is a directional sensor; it only sees what is in front of it.
This requirement—that reconstructed 3D points must reside in front of the camera—is known as the cheirality constraint. While simple to state, it is difficult to implement because it transforms a system of equations (which are easy to solve) into a "semi-algebraic" system of inequalities (which are notoriously hard). As Kileel notes, most solvers ignore these inequalities during the calculation, leading to a surplus of non-physical results that must be pruned later.
Disambiguation via Visibility
The most common mechanism for identifying the "veridical" scene is the application of visibility heuristics. Harit P. Trivedi demonstrated as early as 1986 in On the Reconstruction of a Scene from Two Unregistered Images that while derivations for camera geometry yield multiple solutions, incorrect ones almost always manifest as nonsensical depth values.
Researchers have since refined these checks into specific geometric filters:
- Epipolar Segmentation: Shengjie Li and Yuanxin Wu show in Segmenting Epipolar Line that by combining epipolar geometry with cheirality, one can prune 50% of the search space for feature matching. They prove that a matching point cannot exist on the entire epipolar line, but only on segments that represent points in front of both lenses.
- Surface Normals: When using affine correspondences (local image warps), Daniel Barath and colleagues apply visibility checks to ensure that the reconstructed surface normals don't imply the camera is looking at the "back" of an opaque object (A Minimal Solution for Two-View Focal-Length Estimation Using Two Affine Correspondences).
- Manifold Reduction: Kostas Daniilidis and Roberto Tron use these constraints to simplify the mathematical space itself, reducing the "essential manifold" to a signed version that excludes mirror-image configurations (On the Quotient Representation for the Essential Manifold).
Beyond Vision: Sensing and Symmetry
The principle of using physical "impossibility" to resolve mathematical "possibility" extends beyond traditional cameras. In Bistatic Passive Sensing via CSI Power, Zhongqin Wang and J. Andrew Zhang resolve mirroring ambiguities in Wi-Fi signal tracking by applying the principle that a reflected path must always be longer than a direct line-of-sight path.
Similarly, Kosuke Takahashi uses an orthogonality constraint to calibrate cameras via mirrors. Out of 64 mathematical possibilities, only one satisfies the physical requirement that the vector connecting reflections must be orthogonal to the intersection of the mirror planes (A new mirror-based extrinsic camera calibration using an orthogonality constraint).
From Post-Filtering to Early Pruning
Modern research is moving away from simply "filtering" results toward integrating constraints directly into the solver to improve speed and robustness.
In Lambda Twist, Mikael Persson and Klas Nordberg solve for point depths rather than camera poses. This allows them to immediately reject any solution branch that produces a negative depth, preventing the algorithm from wasting cycles on "ghost" configurations.
Taking this a step further, Petr Hruby and Tomáš Pajdla use machine learning to bypass the search for multiple roots entirely. In Learning to Solve Hard Minimal Problems, they train a neural network to predict a single, physically promising starting point for a numerical solver. By focusing only on the "real" solution from the start, they achieve speedups of up to 1000x, demonstrating that the most efficient way to solve a geometric problem is to never consider the non-physical solutions in the first place.