Lifting Images to Higher Dimensional Orientation Spaces

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 263 papers106 conceptsfrozen 2026-10-04

In computer vision, we often treat images as two-dimensional grids of intensity. While this works for simple shapes, it fails fundamentally when dealing with "filamentous" or elongated structures like blood vessels, neural fibers, or cracks in infrastructure. The problem is the crossing: in a 2D projection, two vessels that pass over one another appear to merge into a single junction. Standard filters, which look only at local spatial neighborhoods, cannot distinguish between a single X-shaped object and two independent lines crossing.

To solve this, researchers have developed a framework to "lift" images into higher-dimensional spaces where orientation is treated as an explicit coordinate rather than just a property of a pixel.

The Mechanism of Lifting

The core idea, pioneered in foundational work like Invertible Orientation Scores as an Application of Generalized Wavelet Theory by Remco Duits et al., is to transform a 2D image $I(x, y)$ into a 3D "orientation score" $U(x, y, \theta)$.

This is achieved by applying a bank of anisotropic (directional) filters—often called "cake wavelets"—at every pixel. If a pixel $(x, y)$ contains a vertical line, the filter corresponding to $\theta = 90^\circ$ will have a high response, while the filter for $\theta = 0^\circ$ will be near zero. By stacking these responses, we create a 3D volume where the third axis represents the local heading of the structure.

From Intersections to Overpasses

The intuition for why this works is best described by Yi Liu et al. in Intersection to Overpass. In a 2D image, a crossing is a point of collision. In the lifted 3D space, however, the two crossing lines have different orientations. Consequently, they are separated along the $\theta$-axis.

What was an intersection in 2D becomes an overpass in 3D. This "disentanglement" allows algorithms to:

The Geometry of Roto-Translations

This lifted space is not just an arbitrary 3D volume; it is the Roto-translation group $SE(2)$ (or $SE(3)$ for 3D images). This group structure is essential because "moving" in this space must respect the relationship between position and angle. If you are at a certain orientation, you can only move "forward" in that direction.

This leads to the use of Sub-Riemannian geometry. In this framework, the "shortest path" between two points is not a straight line but a curve that accounts for bending costs. Gonzalo Sanguinetti, Alexey Mashtakov, and Remco Duits have developed efficient "Fast Marching" algorithms to find these geodesics (Sub-Riemannian Fast Marching in SE(2)). These paths naturally mimic the way the human visual cortex completes contours, as explored in A Cortical Based Model for Contour Completion.

Scaling to 3D and Deep Learning

The concept extends naturally to 3D medical data, such as brain MRI. In Brain Connectivity Measures via Direct Sub-Finslerian Front Propagation, Jorg M. Portegies and colleagues lift 3D spatial data into a 5D space (3D position + 2D orientation on a sphere). This prevents "leakage" where different neural fiber bundles cross, a common error in standard 3D tractography.

Recently, this geometric intuition has been integrated into deep learning. Instead of standard convolutions, PDE-based Group Equivariant CNNs (PDE-G-CNNs) learn to solve partial differential equations directly on these lifted groups. As Bart M. N. Smets and Erik J. Bekkers show in PDE-based Group Equivariant Convolutional Neural Networks, these networks are inherently equivariant to rotations and translations. They don't need to "learn" that a rotated vessel is the same object; the architecture's underlying geometry already knows it.

Papers in this version

Showing 60 of 63 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 54 papers
  2. Version 1 · 2026-10-04 · 63 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_orientation_score_lifting_e81d4ee3,
  title = {orientation score lifting},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/orientation-score-lifting/rdr_e81d4ee367c24b8fb628e66b5b0a4a10},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_lifting_images_to_higher_dimensional_ori_e81d4ee3_v1,
  title = {Lifting Images to Higher Dimensional Orientation Spaces},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/lifting-images-to-higher-dimensional-orientation-spaces/rdr_e81d4ee367c24b8fb628e66b5b0a4a10/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}