When we capture a 3D scene from multiple camera viewpoints, the resulting images are not independent. They are bound by rigid geometric laws. In computer vision, we traditionally express these relationships using multi-view matching tensors: the bilinear fundamental matrix for two views, the trilinear trifocal tensor for three views, and the quadrilinear quadrifocal tensor for four views.
At first glance, as the number of views increases, the mathematical complexity seems to explode. A trifocal tensor has 27 components, while a quadrifocal tensor has 81. Yet, the underlying physical scene has a fixed number of degrees of freedom. This creates a paradox: our geometric representations grow exponentially, but the true information they contain is highly constrained.
The key to resolving this paradox lies in understanding algebraic redundancy. By looking under the hood of multi-view tensors, we can see why higher-order constraints are inherently redundant, how they exhibit low-rank structures, and how we can exploit this compressibility to build robust 3D reconstruction algorithms.
The Joint Image and the Sufficiency of Three Views
To understand why multi-view tensors are so redundant, we must look at how they are generated. In a foundational work, Bill Triggs introduced the "Joint Image" framework in Matching Constraints and the Joint Image. Instead of treating $m$ images as separate 2D spaces, Bill Triggs showed that they can be viewed as a single 3D subspace embedded within a higher-dimensional "joint image space." This subspace, known as the Joint Image Grassmannian, implies that for $m$ images, the geometry is strictly governed by exactly $11m - 15$ algebraically independent components.
Because all matching constraints are merely different algebraic reflections of this single underlying subspace, higher-order tensors do not actually add new information. Faugeras et al. demonstrated this in On the Geometry and Algebra of the Point and Line Correspondences Between N Images by analyzing the rank of the projection matrix that maps 3D points to 2D planes. By enforcing a rank constraint of at most three, they proved that any geometric relation involving four or more images is algebraically dependent on simpler two-view and three-view constraints. For instance, quadrilinear relations can be completely broken down into combinations of bilinear and trilinear tensors.
In fact, Vidal et al. pushed this further in Optimal motion estimation from multiple images by normalized epipolar constraint, showing that if two-view epipolar constraints are properly statistically normalized, higher-order multilinear constraints become entirely redundant for motion optimization.
Internal Constraints and Rank Deficiencies
If higher-order tensors are algebraically dependent, why do we use them? The answer lies in their estimation properties. As Anders Heyden showed in A Common Framework for Multiple View Tensors, higher-order tensors like the quadfocal tensor allow for the direct, linear estimation of multi-view geometry from a minimum number of point correspondences without relying on lower-order initializations.
However, extracting the true camera parameters from these tensors is difficult because they contain internal, non-linear constraints. For example, as Papadopoulo et al. detailed in A New Characterization of the Trifocal Tensor, the trifocal tensor has 27 components but only 18 degrees of freedom. The remaining 9 degrees of freedom are locked behind complex internal algebraic constraints.
These constraints arise because multi-view matching tensors are derived by requiring a matrix of projection rays to be rank-deficient (Gasparini et al. in Multi-view matching tensors from lines for general camera models). Because the physical properties of camera rotation and translation force these high-dimensional tensors to be rank-deficient, they exhibit massive structural redundancy.
To bypass these messy non-linear constraints, Hebert et al. in Coordinate-Free Carlsson-Weinshall Duality and Relative Multi-View Geometry showed that multi-view constraints can be reformulated as "reduced trilinearities" where the coefficients are subject only to simple linear constraints (such as requiring that they sum to zero). Meanwhile, Trager et al. provided an exact counting formula for these independent constraints based on the dimensions of the joint image variety in The Joint Image Handbook.
The Global Punchline: Low-Rank Block Tensors
The realization that local multi-view constraints are redundant leads to a powerful modern breakthrough in global camera synchronization. If we collect the geometric relationships of all camera triplets or quadruplets in a large-scale scene and stack them into a massive "block tensor," what happens to its rank?
Intuitively, one might expect this block tensor's complexity to grow with the number of cameras. However, Miao et al. discovered a remarkable property in Tensor-Based Synchronization and the Low-Rankness of the Block Trifocal Tensor: a block tensor containing all trifocal tensors of a scene maintains a constant, fixed multilinear rank of exactly $(6, 4, 4)$, regardless of how many cameras are added.
Individual Tensors (High-Dimensional, Redundant)
[ Trifocal Tensor 1 ] \
[ Trifocal Tensor 2 ] --> Stacking into Global Block Tensor --> Fixed Multilinear Rank (6, 4, 4)
[ Trifocal Tensor N ] / (Compressible via Tucker Decomposition)
This fixed low-rank property means that multi-view geometry is globally highly compressible. Instead of solving for camera poses through local, incremental updates (which accumulate tracking drift), the entire scene can be solved via global optimization using Tucker factorization.
The authors extended this concept to 4-view geometry in QuadSync: Quadrifocal Tensor Synchronization via Tucker Decomposition, proving that block quadrifocal tensors also admit a low-rank Tucker decomposition. Because the factor matrices of this decomposition correspond directly to the underlying camera matrices, decomposing the stacked tensor yields the camera poses globally and robustly.
Exploiting Redundancy for Solvers
Understanding this algebraic redundancy allows us to build highly efficient algorithms for 3D reconstruction. For instance, Kukelova et al. demonstrated in A Clever Elimination Strategy for Efficient Minimal Solvers how systematic elimination theory can reveal hidden higher-order constraints (such as a fifth-degree constraint on the fundamental matrix). By isolating these constraints, they shift the heavy algebraic lifting to an offline pre-processing step, enabling real-time minimal solvers during RANSAC loop closures.
Furthermore, Hartley et al. showed in Reconstruction from Projections Using Grassmann Tensors that decomposing multi-view projection tensors back into individual camera matrices can be simplified into recovering a matrix from its block-principal minors. This algebraic reduction enables non-iterative, direct estimation of camera setups.
Even in simplified scenarios, like the 1D structure and motion problem analyzed by Kahl et al. in Ambiguous Configurations for the 1D Structure and Motion Problem, the multi-view relationships collapse into a compact $2 \times 2 \times 2$ trilinear tensor. This proves that the underlying low-rank, redundant nature of projective geometry holds true across dimensions, offering a unified mathematical foundation for mapping the physical world from images.
Key Foundational Work
On the Geometry and Algebra of the Point and Line Correspondences Between N Images (1995) — contributed 4 concept elements to this direction.
A Common Framework for Multiple View Tensors (1998) — contributed 2 concept elements to this direction.
Matching Constraints and the Joint Image (1995) — contributed 1 concept elements to this direction.