Training Orientation-Resolved Vessel Segmenters on Synthetic Crossings

Retinal vessels, nerve fibers and roads cross one another. At a crossing, a flat 2D image gives the same pixel to two different structures. A segmentation network trained on such images can say "vessel here" but cannot say which vessel, so crossings get mistaken for branch points and thin vessels are lost where they meet thick ones. A classical fix is to "lift" the image into an orientation score. This is a 3D stack in which each slice responds only to lines at one angle, so the two vessels of a crossing sit on different slices and no longer collide. The trouble is that orientation-score methods have stayed handcrafted, and the data needed to train them is missing. Standard benchmarks provide only flat binary masks, and real crossings are a handful of pixels per image.

This note proposes making that missing supervision synthetically and combinatorially. We take two vessel patches, rotate one, and superimpose them. Because the orientation score is a linear transform of the image, the score of the composite is approximately the sum of the two parts' scores. We also know exactly which part owns which orientation slice. Every pair of patches at every relative angle becomes a crossing with an exact per-orientation label. A few hundred annotated patches therefore turn into tens of thousands of labeled crossings. The core claim is that this orientation-resolved supervision is only usable in the lifted space. The model shares weights across angles, so it ignores global rotation, yet it keeps angle as an explicit coordinate rather than pooling it away. Together these let a learned model separate crossings that flat models, even models trained with the same composites, cannot.

Core Claim and Positioning

Three bodies of work meet here, each in a different role.

The first is group-invariant differential operators, which supply the representation and the operators. Franken and Duits' Crossing-Preserving Coherence-Enhancing Diffusion on Invertible Orientation Scores showed that lifting to the roto-translation group SE(2) lets diffusion run along fibers without blurring their crossings. Robust and Fast Vessel Segmentation via Gaussian Derivatives in Orientation Scores segmented fundus vessels with second-order "left-invariant" derivatives. These are derivatives taken in a frame that rotates with each slice: along the vessel, across it, and across angle. Both pipelines are fixed and unsupervised. The vessel-segmentation pipeline also ends by taking the maximum over orientations, which throws away exactly the information that distinguishes the two vessels at a crossing. We keep their operators but learn how to combine them, and we never collapse the angle axis until evaluation.

The second is the invariance–discriminability tradeoff, which supplies the lens. Making a feature blind to a transformation discards information the task may need. Learning rotational features for filament detection rotated steerable derivative features to a canonical angle before an SVM, which buys invariance by assigning one orientation per pixel. Online Invariance Selection for Local Feature Descriptors chooses between rotation-invariant and rotation-variant descriptors region by region. Affine Steerers for Structured Keypoint Description keeps affine information by steering descriptors equivariantly, but assumes one transformation per patch. None of these can give two different orientations to the same pixel, which is precisely what a crossing requires. Our model is equivariant: rotating the input permutes the slices rather than changing the answer. It is therefore invariant at the level of the final segmentation while staying fully discriminative in angle.

The third is combinatorial augmentation, which supplies the training signal. Mitigating deep double descent by concatenating inputs stacked image pairs to grow the effective sample size quadratically. Advancing Compositional Awareness in CLIP with Efficient Fine-Tuning concatenated image pairs and swapped entities to create compositional hard negatives. Our composites borrow the same move. The difference is that the composition is physically motivated, and it produces a label those methods cannot: an exact assignment of each component to orientation slices.

Planckian Jitter is a constraint on the design. It showed that augmentations ignoring image physics destroy useful features. Accordingly, our compositing follows light absorption rather than naive pixel addition.

Research Program

Compositing. Each fundus image is converted to the green channel. A large-scale background estimate is subtracted in the log-intensity domain. Under Beer–Lambert absorption, overlapping vessels then add approximately, and the background is not counted twice. A vessel component from patch B is masked by its dilated label, rotated by an angle $\phi$, and added to patch A.

For each component $k$, we compute a centerline orientation $\theta_k(x)$ from the tangent of its label skeleton. The orientation-resolved target is

$$T(x,\theta) = \max_k \, M_k(x)\, \kappa\big(\theta - \theta_k(x)\big),$$

where $M_k$ is component $k$'s binary mask and $\kappa$ is a von Mises-shaped bump whose width matches the angular bandwidth of the lifting wavelets. With $N$ training patches and $R$ sampled relative angles this yields $N^2 R$ labeled crossings. We sample relative angles stratified toward the acute range, where flat methods fail most.

Model. The input is lifted with fixed, invertible cake wavelets into 16 orientation slices. The network is a stack of learned layers. Each layer forms a linear combination of left-invariant Gaussian derivatives up to second order, at several scales, followed by a pointwise nonlinearity. Weights are shared across slices, so the network is roto-translation equivariant by construction, and it outputs a predicted score $\hat U(x,\theta)$.

Training on two kinds of data. On composites, the loss is per-slice cross-entropy against $T$. On real, uncomposited images, a crossing's slice ownership is unknown. There we apply the per-slice loss only away from skeleton junctions. At every pixel we also apply an ordinary loss on the max-over-angle projection, so that flat-mask supervision is still used.

Label noise. The file flags that pairing propagates label noise. We therefore repeat training under the synthetic label erasure protocol of Study Group Learning to measure whether composites amplify erased labels.

Baselines

The naive reference point is a standard 2D UNet with random rotation, flip and scale augmentation. It is trained on DRIVE, STARE and CHASE_DB1 with the same epochs and the same hyperparameter search budget (an equal number of trials) as our model, and with parameter count matched within 10%.

The strongest unsupervised orientation-score method is the left-invariant Gaussian-derivative pipeline of Robust and Fast Vessel Segmentation via Gaussian Derivatives in Orientation Scores. It reports DRIVE accuracy 0.9446, sensitivity 0.7744 and specificity 0.9708, and STARE accuracy 0.9511, sensitivity 0.7940 and specificity 0.9707. We rerun it to obtain the metrics it does not report.

The strongest learned method in the file is Study Group Learning. It reports a DRIVE AUC of 0.9857 with K=8 under label erasure r=0.5. We compare to it under that same noisy protocol, and also rerun it on clean labels. For CHASE_DB1 the file gives no reported number for this method, only qualitative gains, so we rerun it there.

For crossing-level metrics, no reported result was found on Lacuna for any method. All rows in that column will be our own reruns.

Three ablations isolate the key ingredients. The first is the Cartesian UNet trained on the identical composites with union labels. This tests whether compositing alone, without lifting, explains the gains. The second is our lifted model without composites, which tests whether lifting alone suffices. The third is our lifted model with composites but trained only on the max-projected union label, which tests whether orientation-resolved supervision is the active ingredient.

The bar for beating the field has two parts. On crossing metrics, the full model must exceed both the Cartesian-composite ablation and SGL by a margin larger than the spread across three seeds. On global AUC, it must stay within that seed spread of SGL.

Why not just…?

Why not just do copy-paste compositing with an ordinary UNet? This is the real threat. Synthetic crossings may teach a flat network to keep thin vessels continuous through junctions without any lifting. Our hypothesis is that a flat output cannot represent two owners at one pixel, so the benefit of compositing should saturate at "vessel present" without improving crossing-versus-bifurcation decisions. The Cartesian-composite ablation settles this directly. If it matches the lifted model on the junction metrics, we will report that lifting is unnecessary and drop the claim.

Why not just use the handcrafted orientation-score filter with a learned threshold? Its authors note it struggles with pathology and needs adaptive thresholding. Learning rotational features for filament detection found that handcrafted ridge filters fail on structured background, which learned features reject. Our ablation without composites is essentially this filter made learnable, so it measures how much learning adds before compositing is introduced.

Why not just use mixup-style blending or input concatenation? Blending averages labels and concatenation places images side by side. Neither creates a geometric crossing with known angle and ownership, which is the specific signal this proposal relies on. We include a mixup-trained lifted model as an extra row. We expect it to help global AUC slightly and junction metrics not at all.

Proposed Experiments

Synthetic recovery test. The first check runs on held-out composites built from test-split patches, where ground-truth ownership is exact. We measure per-component angular error and the fraction of crossings whose two components are recovered on separate slices, plotted as a function of relative crossing angle. The full model should degrade gracefully down to angles near the wavelet bandwidth, while the Cartesian ablation cannot report ownership at all.

Real crossing test. The decisive real-data test requires junction annotations that the benchmarks do not provide. We will hand-label every vessel junction in the DRIVE and STARE test images as a crossing or a bifurcation, with two annotators and reported agreement. Each method's output is then tracked through each junction by following its two highest-scoring orientation slices. We score crossing-versus-bifurcation accuracy, and thin-vessel sensitivity within a fixed radius of each junction.

Global and noise checks. We report AUC, accuracy, sensitivity and specificity on all three benchmarks, and repeat the comparison under SGL's r=0.5 erasure protocol.

Abandonment criteria. We will abandon the core claim if either of two results holds. One is that the Cartesian-composite ablation is within seed noise of the full model on junction accuracy. The other is that composites under label erasure lower AUC more than they do for the uncomposited lifted model; this would mean quadratic noise propagation outweighs the gain.