Overcoming Triangulation Instability in Small Baseline Motion

38 concepts33 papers1 proposal

To understand why small camera displacements cause traditional 3D reconstruction algorithms to fail, imagine trying to estimate the distance to a faraway mountain peak. If you take one large step to the side, the peak's position relative to the foreground shifts noticeably. This shift is parallax, and measuring it is the core mechanism of triangulation.

Now imagine taking a step of only a single millimeter. The visual shift is imperceptible, buried beneath the limits of your visual acuity. If you try to calculate the distance mathematically using this tiny movement, even a microscopic error in estimating your position or matching pixels between views will cause your calculated distance to swing by miles.

In computer vision, this is the small baseline problem. When the distance between camera viewpoints (the baseline) approaches zero, traditional geometric pipelines encounter severe mathematical instability.

The Geometry of Triangulation Collapse

In standard Structure from Motion (SfM) and visual SLAM, a 3D point is reconstructed by finding the intersection of rays projected from different camera positions. Mathematically, the estimated depth $d$ is inversely proportional to the parallax angle $\theta$ between these intersecting rays.

When a camera moves minimally—such as during a slow video pan, a drone hovering in place, or the natural tremor of a human hand—the parallax angle $\theta$ approaches zero. Because depth is inversely proportional to the sine of this angle, the sensitivity of depth estimation explodes exponentially as the angle shrinks. As the authors of Triangulation: Why Optimize? demonstrate, minor errors in pixel matching or pose estimation are amplified catastrophically, leading to extreme depth uncertainty.

This geometric degeneracy manifests across several common real-world scenarios:

The Asymmetry of Rotation and Translation

When a camera undergoes a small displacement, the resulting apparent motion of pixels (optical flow) is a combination of two components: rotational flow and translational flow.

$$\text{Optical Flow} = \text{Rotational Flow}(\omega) + \text{Translational Flow}(t, d)$$

Crucially, rotational flow depends entirely on the camera's angular rotation $\omega$ and is completely independent of scene depth. Translational flow, however, depends directly on both the translation vector $t$ and the depth $d$.

As the baseline shrinks toward zero, the translational flow signal drops below the image noise floor, making the translation direction vector mathematically unrecoverable. However, the rotational signal remains unaffected. This creates a stark mathematical asymmetry: even when a baseline is too small to resolve translation or depth, it can still yield highly accurate relative rotation.

Fredrik Kahl et al. exploited this property in Non-sequential structure from motion, showing that incorporating these narrow-baseline pairs stabilizes the global rotation network of an SfM pipeline.

Conversely, if an algorithm attempts to force a translation calculation from these degenerate pairs, it injects massive noise into the system. As a result, global coordinate averaging techniques can fail entirely. To protect global reconstructions from this corruption, Xin Wang et al. developed a filtering framework in Robustifying Relative Orientations With Respect to Repetitive Structures and Very Short Baselines for Global SfM that identifies and isolates narrow-baseline pairs, discarding their unstable translation vectors while preserving their robust rotational constraints.

Turning Hand Tremor into a Micro-Baseline Signal

While traditional algorithms treat sub-millimeter camera displacement as a nuisance or noise to be filtered out, a major branch of recent research flips this perspective: unintentional hand tremor can be leveraged as a dense, continuous geometric signal.

When a user takes a burst of photos or a short video clip, involuntary hand shake creates a sequence of "micro-baselines." Because these displacements are so small, traditional discrete feature matching fails; the pixel displacement is simply too close to the noise floor (Ilya Chugunov et al. in Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography).

To extract depth from these micro-baselines, researchers shift from discrete feature geometry to continuous photometric analysis. Instead of matching sparse corners, algorithms track how pixel intensities change continuously across the image sequence. Hyowon Ha demonstrated in High Quality Structure from Small Motion for Rolling Shutter Cameras and High-Quality Depth from Uncalibrated Small Motion Clip that analyzing dense intensity profiles across a small motion clip allows for the recovery of fine surface details that escape wide-baseline stereo matching.

Modern approaches supercharge this concept by using coordinate-based neural networks and implicit fields. By modeling pixel trajectories continuously, these frameworks aggregate the weak parallax signal from dozens of micro-views. For example, Ilya Chugunov et al. showed in The Implicit Values of A Good Hand Shake: Handheld Multi-Frame Neural Depth Refinement and Neural Spline Fields for Burst Image Fusion and Layer Separation that parameterizing camera motion via continuous splines or neural implicit functions allows the system to distill millimeter-scale hand tremors into high-fidelity, centimeter-scale depth maps.

Regularization and Alternative Parameterizations

When physical motion cannot provide a reliable baseline, the underlying optimization problem becomes mathematically ill-posed. To prevent geometric collapse, the system must be constrained using structural priors or alternative mathematical representations.

Structural and Semantic Priors

One strategy is to bound the optimization using environmental constraints. In indoor environments, Alejo Concha et al. introduced Manhattan-world and piecewise-planar constraints in Manhattan and Piecewise-Planar Constraints for Dense Monocular Mapping, forcing depth solutions to adhere to flat surfaces when parallax is absent.

In modern pipelines, this structural guidance often comes from deep learning models trained on vast datasets. Sheng Liu 0017 et al. demonstrated in Depth-Guided Sparse Structure-from-Motion for Movies and TV Shows that integrating single-frame depth priors from pretrained neural networks can anchor tracking and scene geometry in cinematic videos where camera motion is too subtle for standard SfM.

Mathematical Re-parameterization

An alternative strategy changes how the geometry is modeled to bypass the $1/\sin\theta$ singularity entirely:

By shifting from discrete, wide-baseline geometric intersections to continuous photometric tracking, structural priors, and neural implicit fields, modern computer vision transforms the small baseline problem from a catastrophic failure mode into a source of high-fidelity spatial detail.

Key Foundational Work

Estimation of Epipolar Geometry from Apparent Contours: Affine and Circular Motion Cases (1999) — contributed 2 concept elements to this direction.
Differential Matching Constraints (1999) — contributed 1 concept elements to this direction.
Non-sequential structure from motion (2011) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 32 papers
  2. Version 1 · 2026-10-04 · 35 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_small_baseline_instability_63fb0891,
  title = {small baseline instability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/small-baseline-instability/rdr_63fb08911dd743a2b5d031dafb895027},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_small_baseline_instability_63fb0891_v2,
  title = {small baseline instability},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/small-baseline-instability/rdr_63fb08911dd743a2b5d031dafb895027/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake