When we project a three-dimensional world onto a two-dimensional image, we inevitably lose depth information. For curvilinear or filamentous networks—such as blood vessels in an X-ray, neural pathways, roads in satellite imagery, or strokes in calligraphy—this loss of depth manifests as a severe geometric bottleneck: topological ambiguity.
When two independent filaments cross each other in 3D space, their 2D projection forces them to share the exact same spatial coordinates $(x, y)$. To a standard image processing algorithm operating purely on a flat pixel grid, this visual crossing is indistinguishable from a true branching junction (a bifurcation). This structural overlap causes conventional tracking, segmentation, and enhancement algorithms to fail, leading them to erroneously merge distinct paths or jump tracks entirely.
2D Spatial Projection (Ambiguous Crossing)
\ /
\ /
--X-- <-- Spatial overlap mixes orientations at this point
/ \
/ \
3D Lifted Orientation Space (Disentangled Overpass)
/
/ (Orientation θ_1)
|
-------|------- (Orientation θ_2)
|
/
/
To resolve this ambiguity, a powerful paradigm has emerged: lifting the image into a higher-dimensional space where orientation acts as an explicit extra coordinate. By transforming an intersection into an overpass, this approach allows algorithms to untangle complex structural networks with mathematical precision.
The Bottleneck of Spatial Overlap
The core challenge of topological ambiguity spans numerous domains. In medical imaging, Hildegard Koehler et al. demonstrate that 2D X-ray projections of 3D coronary artery trees frequently conflate visual crossings with true vessel bifurcations, corrupting the mathematical graphs used to model blood flow (Analyse des Herzkranzgefäßbaums...). This projection-induced occlusion severely complicates automated tracking across time-series sequences, as noted by Seung Yeon Shin et al. (Extraction of Coronary Vessels...) and Peng Zhou (A framework of myocardial bridge detection...).
The issue is not unique to 3D-to-2D projections; it occurs whenever independent linear features intersect on a flat plane. Traditional image filters rely entirely on local spatial gradients. At an intersection point, these gradients become ill-defined, causing "orientation mixing." As Erik Franken et al. observe, this mixing introduces blurring or checkerboard artifacts because the filter cannot simultaneously represent multiple directional components at a single coordinate (Nonlinear Diffusion on the 2D Euclidean Motion Group).
Whether trying to trace retinal vessel trees (Huiqi Li et al., /paper/art_c06ee81891a046df9395a188957fe556), reconstruct 3D wireframe models from single urban images (Siddhant Ranade et al., /paper/art_131770f35a194f6abde545029ce22d72), count overlapping cells in embryo development videos (Yuqi Cui et al., /paper/art_66c3f78aa1be42ce83a032b57770d653), or isolate overlapping strokes in calligraphy (Meng Li et al., /paper/art_7501c558289c4e6799009524d826ddf5), purely spatial analysis lacks the context required to determine which segment belongs to which underlying path.
The Intuition of Domain Lifting
To understand why lifting works, imagine a standard 2D image of two crossing lines. At the exact intersection point $(x_0, y_0)$, the pixel data is a combined, ambiguous mass. However, if we evaluate the image using a bank of orientation-selective filters (similar to how the primary visual cortex operates), we can extract an extra piece of information: the angle $\theta$ of the structures passing through that point.
By appending this orientation angle as a new axis, we map the 2D image into a 3D space parameterized by $(x, y, \theta)$. In this "lifted" space, the two crossing lines no longer collide. The first line passes through $(x_0, y_0, \theta_1)$ and the second through $(x_0, y_0, \theta_2)$. Because $\theta_1 \neq \theta_2$, the paths are spatially segregated.
Mickaël Péchaud et al. demonstrated that treating images as higher-dimensional volumes (incorporating position, orientation, and even radius) allows path-finding algorithms to navigate complex intersections seamlessly without jumping between distinct structures (Extraction of tubular structures over an orientation domain). Similarly, Jiong Zhang et al. showed that this cortical-inspired lifting prevents errors when calculating critical morphological biomarkers like vessel length and tortuosity (Brain-inspired algorithms for retinal image analysis).
Mathematical Formalization via Lie Groups
To process these lifted images without destroying their geometric properties, researchers formalize the lifting using continuous groups—specifically the roto-translation group $SE(2)$ or the similitude group $SIM(2)$, which accounts for translation, rotation, and scaling (Upanshu Sharma, /paper/art_d8bdd2261b8449699a166568416c497e).
The foundational framework for this is the invertible orientation score, developed by Remco Duits et al. (Invertible Orientation Scores...). The pipeline operates in three distinct phases:
- Lifting (Orientation Score Construction): The 2D image is decomposed into a series of orientation-dependent layers using specialized wavelets or directional filters.
- Left-Invariant Processing: Image enhancement or segmentation is performed within this higher-dimensional domain. Because the operations are left-invariant (meaning they commute with rotations and translations), the processing respects the underlying geometry of the lines. For instance, Erik J Bekkers et al. apply left-invariant Gaussian derivatives to track fine vascular structures through crossings (Robust and Fast Vessel Segmentation...), while Bart M. N. Smets et al. use left-invariant partial differential equations (PDEs) for nonlinear denoising without blurring crossing boundaries (Total Variation and Mean Curvature PDEs...).
- Projection: The enhanced orientation score is reconstructed back into a standard 2D image without any loss of topological information.
This framework is highly adaptable. For example, Michiel H. J. Janssen extended this concept to 3D medical volumes, lifting them into a 6D position-orientation space to untangle intersecting 3D nerve fibers and complex vascular networks (Design and Processing of Invertible Orientation Scores of 3D Images).
Geodesic Tracking and Pathfinding
Once an image is lifted into an orientation score, tracking a vessel or filament reduces to finding the shortest path—a geodesic—within this higher-dimensional space. However, standard Euclidean distance metrics do not apply because moving sideways relative to a filament's orientation is physically impossible for a continuous line.
To enforce realistic continuity, researchers implement sub-Riemannian or Finsler metrics that penalize sharp changes in direction. Gonzalo Sanguinetti et al. developed a highly efficient sub-Riemannian Fast Marching algorithm in $SE(2)$ that restricts path travel along the extracted orientation angles, preventing tracking algorithms from "leaking" into crossing structures (Sub-Riemannian Fast Marching in SE(2)).
Allowed Movement (Low Penalty) Forbidden Movement (High Penalty)
-----> -----> ^
/ |
(Smooth alignment with θ) (Sharp lateral jump)
Recent advancements have made these tracking metrics data-driven. Nicky J. van den Berg et al. introduced a "Cartan connection" framework that dynamically adjusts the tracking metric based on local image context, ensuring cusp-free, continuous tracking across entire vascular trees (Geodesic Tracking via New Data-Driven Connections...). This geometric tracking has also been adapted by Nicky J. van den Berg to operate directly on spherical geometries, eliminating the distortions caused by mapping the spherical human eye onto a flat plane (Crossing-Preserving Geodesic Tracking on Spherical Images).
Learning-Based and Layered Alternatives
While differential geometry offers an elegant analytical solution, modern deep learning approaches tackle topological ambiguity by embedding orientation awareness directly into neural network architectures.
Standard segmentation foundation models often fail in these scenarios; as Jun Ma notes, spatial prompts like bounding boxes naturally overlap when branching structures intertwine, causing the model to confuse adjacent tissues (Segment anything in medical images). Similarly, standard loss functions struggle to enforce connectivity when 2D projections create occluding masses (Doruk Oner et al., /paper/art_3f763129095d4d0e8d5f5a9daccbd7d9).
To overcome this, Yi Liu et al. designed an "Intersection to Overpass" framework using an orientation-aware neural network with multiple directional branches. This network explicitly separates crossing filaments into independent layers before a pairing algorithm stitches the corresponding fragments back into individual instances (Intersection to Overpass...).
Neural Orientation Layers
Layer 1 (Horizontal Components): -------------------
Layer 2 (Vertical Components): |
|
Stitched Instance Output: | Overpass Resolved
Other deep learning methods bypass static segmentation entirely in favor of iterative tracking. Chao Liu et al. developed DeepBranchTracer, which uses Long Short-Term Memory (LSTM) networks to iteratively trace curvilinear structures, utilizing historical path context to maintain branch identity through dense intersections (DeepBranchTracer...).
By moving beyond flat spatial representations—whether through the rigorous mathematics of Lie groups or the contextual memory of deep networks—modern computer vision can successfully untangle the complex, overlapping topologies of the physical world.
Key Foundational Work
Analyse des Herzkranzgefäßbaums für die prä- und post-operative Diagnose (2004) — contributed 3 concept elements to this direction.
Dendrite Tracking in Microscopic Images using Minimum Spanning Trees and Localized E-M (2006) — contributed 2 concept elements to this direction.