Imagine a 2D image containing two crossing blood vessels or neural fibers that form an "X". At the exact intersection point, a standard pixel grid records only a single intensity value. Standard image processing operations, like Gaussian blurring or edge tracking, look at local spatial neighborhoods. At the junction, the gradients from both structures collide, causing traditional filters to blur them together or tracking algorithms to "jump tracks" and follow the wrong vessel.
This limitation stems from a fundamental geometric bottleneck: the standard spatial domain ($\mathbb{R}^2$ or $\mathbb{R}^3$) cannot natively represent multiple directional features at a single point. To resolve this ambiguity, we can "lift" the image into a higher-dimensional space where orientation is treated as an explicit coordinate axis.
The Intuition of the Overpass
To understand why lifting works, consider a highway intersection. If two roads cross at the same grade, traffic must collide or stop. However, if we build an overpass, we add a third dimension—height—allowing cars to pass over each other seamlessly.
Orientation score lifting does exactly this for image data. For a 2D image, it transforms the domain from the flat Euclidean plane $\mathbb{R}^2$ into a 3D space of positions and orientations, often modeled as the Euclidean motion group $SE(2)$ or a homogeneous space $M_2$.
As established by Remco Duits et al. in Invertible Orientation Scores as an Application of Generalized Wavelet Theory, this transformation decomposes the image into an "orientation score" using directional wavelets. Crucially, this process is designed to be invertible, meaning we can process the data in the higher-dimensional space and project it back down without losing information. In this lifted space, the two crossing lines no longer intersect; they pass over each other at different "heights" (angular coordinates).
Michiel Janssen extended this concept to 3D spatial data in Design and Processing of Invertible Orientation Scores of 3D Images, mapping 3D structures into higher-dimensional orientation spaces to decouple overlapping elongated features based on their 3D directional alignment.
Geometry-Aware Regularization and Filtering
Once an image is lifted, we can apply partial differential equations (PDEs) or filters that respect the underlying geometry of the manifold. Instead of diffusing information uniformly in all spatial directions, we diffuse information along the detected orientation.
In Crossing-Preserving Coherence-Enhancing Diffusion on Invertible Orientation Scores, the authors demonstrated that performing nonlinear diffusion on the Euclidean motion group $SE(2)$ allows individual fibers to be enhanced along their respective orientations without cross-blurring at junctions. Similarly, Finn M. Sherry et al. extended diffusion-shock filtering to the space $M_2$ in Diffusion-Shock Filtering on the Space of Positions and Orientations, enabling the distinct processing and inpainting of crossing lines.
To ensure these operations remain accurate under high spatial anisotropy, Gijs Bellaard et al. analyzed sub-Riemannian kernels in Analysis of (sub-)Riemannian PDE-G-CNNs, showing how proper sub-Riemannian metrics prevent approximation failures and improve line-completion performance.
Geodesic Tracking Without Shortcuts
Tracking tubular structures like blood vessels or neural tracts is a major application of this framework. In standard 2D tracking, minimal-path models often take "shortcuts" across sharp corners or noise because they only minimize spatial distance.
By lifting the problem, second-order geometric properties like curvature can be reformulated as first-order properties on the lifted manifold. As shown by Da Chen et al. in A New Finsler Minimal Path Model with Curvature Penalization for Image Segmentation and Closed Contour Detection, adding an orientation dimension allows complex bending penalties to be solved using standard geodesic frameworks.
For automated pathfinding, Gonzalo Sanguinetti et al. introduced a highly efficient numerical approach in Sub-Riemannian Fast Marching in SE(2), allowing tracking algorithms to resolve intersections seamlessly. This has been widely applied to medical imaging; for instance, Nick J. van de Berg et al. leveraged this in Geodesic Tracking of Retinal Vascular Trees with Optical and TV-Flow Enhancement in SE(2) to track complex vascular networks from low-quality clinical scans without getting lost at bifurcations.
Integration into Deep Learning
The principles of orientation lifting have also revolutionized geometric deep learning. Traditional Convolutional Neural Networks (CNNs) are translation-equivariant but struggle with rotations. Group Convolutional Neural Networks (G-CNNs) solve this by incorporating a lifting layer that projects standard input features into a higher-dimensional group representation containing orientation information.
Erik J Bekkers formalized a flexible framework for this in B-Spline CNNs on Lie groups, using B-splines to build networks equivariant to continuous transformation groups by disentangling the identity of a feature from its pose. In Spherical CNNs, Taco Cohen et al. applied a similar lifting mechanism to spherical data, transforming features from a 2D spherical surface to a higher-dimensional rotation group manifold to natively capture 3D rotations.
By treating orientation as an explicit coordinate rather than an implicit feature channel, these architectures bake fundamental geometric priors directly into the network structure.