In geometric computer vision, we often want to solve for the "big picture" all at once. Whether we are reconstructing a 3D scene from thousands of internet photos (Global Structure-from-Motion), mapping an environment with a robot (SLAM), or determining the structure of a protein (Cryo-EM), the goal is the same: find a global configuration of cameras and points that best explains our local measurements.
The "global" approach is elegant and fast, but it has a fatal flaw: it is incredibly fragile. While incremental systems can filter out bad data as they grow, global solvers consider every measurement simultaneously. This means a single incorrect feature match or a false-positive loop closure can act like a "poison pill," distorting the entire reconstruction.
The $L_2$ Trap: Why Least-Squares Fails
Most classical optimization algorithms use a least-squares ($L_2$) objective. This is mathematically convenient because it is differentiable and has a clear probabilistic interpretation under Gaussian noise. However, as Arthur Szlam and Gilad Lerman point out, squaring the error makes the solver hyper-sensitive to outliers.
Imagine a pose graph where nodes are camera positions and edges are relative rotations. If most edges have an error of 1 unit, but one "outlier" edge has an error of 10 units, the $L_2$ solver sees that outlier as a cost of 100. To minimize the total cost, the solver will "stretch" the rest of the graph to accommodate that one bad measurement. This sensitivity is a recurring theme across the field, from 3D face reconstruction to human pose estimation.
Attenuating the Error: Robust Cost Functions
To fix this, researchers move away from pure least-squares toward robust cost functions. The intuition is to "cap" the influence of any single measurement.
Khurrum Aftab and Richard Hartley demonstrated that standard solvers like Levenberg-Marquardt can be adapted to minimize $L_q$ or Huber losses by applying an "attenuation factor" to the error vector. Similarly, Kaishva Chintan Shah and colleagues found that using an $L_1$-norm (which penalizes absolute error rather than squared error) allows for high-fidelity 3D reconstructions even in the extreme noise of Cryo-EM. By not squaring the error, these methods ensure that a massive outlier only exerts a linear, manageable pull on the final solution.
Geometric Consistency and the Logic of Loops
If we can't trust our measurements, we can at least trust geometry. A powerful tool for identifying outliers is cycle consistency.
In a perfect world, if you calculate the relative rotation from Camera A to B, then B to C, and finally C back to A, the result should be the identity matrix (a "zero" rotation). If it isn't, there is an error in that loop. Arman Karimian and Roberto Tron leverage this by using Dual Decomposition (ADMM) to identify rotational outliers before the main optimization even begins.
Taking this further, Shaohan Li and Gilad Lerman introduced Cycle-Sync, a message-passing framework that exploits these loop constraints to remain robust even when 80% of the input data is corrupted. By treating the view-graph as a system of consistent cycles, the algorithm can mathematically "vote out" the edges that don't fit the global topology.
Filtering and Uncertainty-Aware Backends
Another strategy is to prevent outliers from ever reaching the optimizer. Ryun-Seok Kim and Hyukdoo Choi proposed "selective measurement" for GraphSLAM, filtering constraints based on statistical likelihood. In modern LiDAR systems, Xiangcheng Hu and his team use an uncertainty-aware backend (UPGO) that automatically downweights suspicious constraints, preventing "catastrophic map warping" caused by false-positive loop closures.
Even in the front-end, new techniques are emerging. Shengze Jin and Marc Pollefeys use diffusion models to denoise matching matrices, while Luwei Yang and Ping Tan utilize neural networks for Multi-Source Propagation, replacing brittle initialization trees with learned, robust alternatives.
The Goal: Certifiable and Initialization-Free Solvers
The "holy grail" of this research direction is a solver that is both robust to outliers and guaranteed to find the best possible solution without needing a good starting guess.
Traditional Bundle Adjustment is notorious for getting stuck in local minima unless initialized perfectly. Christopher Zach and Je Hyeong Hong addressed this with pOSE, an objective function that allows reconstruction to converge even from random starting points. More recently, Javier Civera and Javier Tirado-Garín introduced C2P, a framework that combines outlier rejection with certifiable optimality.
By integrating robustness directly into the core optimization—rather than treating it as a separate preprocessing step—these researchers are moving toward a future where 3D reconstruction is as reliable as it is fast.