A segmentation network trained on twenty scans has to learn two things at once: what a coronary artery looks like, and which nearby voxels count as "the same structure" when it pools evidence. Current small-data designs fix the second part crudely. NA-UNETR restricts attention to a small cube around each voxel, which keeps the model from latching onto distant, spurious correlations. Yet a vessel only a few voxels wide, running diagonally through that cube, fills a small fraction of it. The rest is surrounding fat, muscle and noise. This is the old point from Analyzing Oriented Patterns, which showed that isotropic operators (ones that treat every direction the same) average across the grain of an oriented structure as readily as along it.
We propose to decide the shape of each voxel's attention neighborhood with a fixed, non-learned estimate of local orientation. The neighborhood becomes a thin ellipsoid laid along the vessel where the image looks tubular, and stays a cube where it does not. The token budget is the same as the cube's, and no new learnable parameters are added. The claim is specific and falsifiable. Under severe data scarcity, putting directional knowledge into where the network aggregates should improve vessel continuity more than putting the same knowledge into what the network sees as an extra input channel. That advantage should also shrink as the training set grows, because with enough data the network can learn orientation-dependent pooling itself.
Core Claim and Positioning
The closest existing work places directional structure in the features. LDAN replaces the first layers of a classifier with a fixed shearlet filter bank. A shearlet is a directional multiscale filter whose support is long and thin, with width roughly equal to length squared. LDAN reports an 80.83% parameter reduction over WaveMix on CIFAR10 and sparser feature maps (sparsity ratio 0.5671 versus roughly 1.0 for wavelets). Two things separate LDAN from this proposal. It works on 2D natural-image classification, and its downstream mixing of features stays isotropic. O²former adds an orientation operator for radiograph super-resolution with 500 training images, but that operator only measures variance along the horizontal and vertical axes. NA-UNETR supplies the small-sample recipe we build on: ImageCAS pretraining, LoRA fine-tuning (training only small low-rank weight updates) on 20 LAD-SEG scans, and local attention. Its windows are strictly cubic.
None of these three changes the geometry of aggregation per voxel according to measured image structure. That change is the whole contribution. We reuse NA-UNETR's backbone and training protocol unchanged.
The orientation estimate comes from classical tubular analysis. Tubular anisotropy for 2D vessel segmentation uses Optimally Oriented Flux (OOF) to recover a vessel's direction and radius. OOF is a filter response that measures how much image gradient flows into a sphere of given radius, and yields a local direction and radius. That work pairs OOF with manual seeds and fast marching. We use only the descriptor, computed densely and fixed before training. Discrete shearlet frames (Construction of Compactly Supported Shearlet Frames) serve as an alternative orientation estimator, so the result does not hinge on one filter family.
The theoretical motivation comes from Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks. It shows that networks beat linear smoothers precisely on functions that vary sharply across a direction and smoothly along it. That result only covers shallow networks, so we treat it as motivation rather than a guarantee.
I-STAR uses "anisotropy" in a different sense: the spread of latent token embeddings in language models. We note it only to avoid confusion; it plays no role here.
Mechanism
At each voxel $x$, the fixed OOF filter bank returns a unit direction $v(x)$, a radius $r(x)$, and a tubularity confidence $c(x) \in [0,1]$. The attention neighborhood is the set of integer offsets $\delta$ that satisfy
$$ \frac{(\delta^\top v)^2}{L(x)^2} + \frac{\lVert \delta - (\delta^\top v)v \rVert^2}{W(x)^2} \le 1 . $$
Here $L(x)$ and $W(x)$ are the half-length and half-width of the ellipsoid. Both are chosen so that the ellipsoid holds the same number of voxels as NA-UNETR's $k \times k \times k$ cube, which keeps compute and token count matched. The aspect ratio interpolates between 1 (a cube-equivalent ball) and a parabolic ratio, driven by $c(x)$ and tied to $r(x)$. Tokens are gathered by trilinear sampling on an oriented grid. Attention weights inside the neighborhood remain learned exactly as in NA-UNETR.
The window shape itself adds zero trainable parameters. We keep it that way deliberately: Thrifty NAS shows that every learned degree of freedom is a liability at small N.
Research Program
The work has three parts. First, we build the steered-attention layer as a drop-in replacement for NA-UNETR's neighborhood attention, with both OOF-based and shearlet-based orientation estimators.
Second, we run a controlled sample-size study. ImageCAS has 1,000 annotated volumes, so we can subsample training sets of 10, 20, 50, 100 and the full set, with three random draws per size, all scored on one fixed held-out test split. This yields learning curves rather than a single point, which is the right way to test a claim about small-sample behavior.
Third, we transfer to the true target regime: ImageCAS pretraining followed by LoRA fine-tuning on the 20 non-contrast LAD-SEG scans, with the same folds and loss as NA-UNETR. PROMISE12 prostate MRI serves as a negative control. The gland is a compact blob with no dominant orientation, so the method's confidence gate should keep windows near-isotropic and the gain should be near zero. A gain there would suggest the effect comes from something other than orientation.
Baselines
The naive reference point is nnU-Net trained directly on each data regime with its default augmentation. NA-UNETR reports 42.54% Dice for nnU-Net on LAD-SEG.
The strongest existing method is NA-UNETR itself, reproduced under its published protocol. It reports 45.64% Dice and a 95th-percentile Hausdorff distance (HD95) of 38.16 mm on LAD-SEG, and 79.49% Dice on full ImageCAS. It is the strongest competitor because it already combines pretraining, parameter-efficient fine-tuning and local inductive bias, the three things most likely to explain any gain we see. We run it ourselves on identical folds, with the same epochs, the same hyperparameter grid (learning rate, LoRA rank, window size $k$) and the same seeds. Beating it means improving HD95 and vessel-break counts by more than the spread across seeds, not just matching Dice.
On PROMISE12, published points are Thrifty NAS at 87.69% Dice and CDA-Net at 92.88%, on different splits. We report our own isotropic-versus-steered pair under one split and do not claim to beat either number.
Three ablations isolate the key ingredient. The first is the most important: a vesselness-channel baseline. It feeds the same OOF direction, radius and confidence maps as extra input channels to NA-UNETR with cubic windows. The network then has identical information but isotropic aggregation, which directly separates "what it sees" from "where it looks." The second is random-orientation steering, which uses the same elongated ellipsoids pointed in random directions. It tests whether the benefit comes from correct orientation or from elongation alone. The third is axis-aligned strips, meaning three orthogonal elongated windows in the spirit of O²former. It tests whether continuous orientation matters.
A fourth variant replaces the fixed OOF orientation with orientation predicted by a small learned head. It asks whether "non-learned" is essential at small N. We expect it to win at the full ImageCAS size and lose at N of 10 to 20.
Why not just…?
Why not just add a vesselness filter as an input channel? This is the most serious threat. It is cheap and gives the network the same orientation information. Our argument is that an input channel leaves pooling isotropic, so evidence for a faint vessel is still averaged with the tissue around it, exactly Kass's failure mode. That is an argument, not evidence, so the vesselness-channel ablation is the central comparison of the study. If it matches steered attention on HD95 and vessel breaks at N of 20 or fewer, the aggregation claim is wrong and we will say so.
Why not just use heavier rotation and elastic augmentation? Augmentation teaches a network to recognize vessels at many angles, but it does not change the fact that each voxel pools over a cube. The file's evidence here is indirect. LDAN's Wide ResNet falls to 55.23% under 90° rotation against LDAN's 68.62%, and Thrifty NAS shows that 9M-parameter models overfit small cohorts despite augmentation. All arms of our study use the same nnU-Net augmentation, so any gain is measured on top of it.
Why not just shrink the network, as Thrifty NAS does? Capacity reduction is complementary rather than competing, and steering adds no parameters. We include a reduced-width NA-UNETR matched to a smaller budget. If it closes the gap at N of 10, the benefit is capacity, not geometry.
Why not just probe a frozen pretrained backbone? FEDD reaches 81% dermatology accuracy with 10% of the data using frozen diffusion features. Its authors note that deep frozen layers lose fine spatial detail, and fine spatial detail is exactly what a few-voxel vessel requires. NA-UNETR's LoRA baseline already covers the fine-tuning side of this trade-off.
Decisive Checks
We report Dice, HD95, and two continuity measures: the fraction of the ground-truth centerline that is covered, and the number of disconnected predicted vessel segments along the LAD. We use continuity measures because Dice barely penalizes a vessel broken into pieces.
Four outcomes would support the claim. Steered attention beats both NA-UNETR and the vesselness-channel baseline on HD95 and segment count at N of 20 or fewer. Its margin over isotropic attention shrinks monotonically toward zero as ImageCAS training size grows to 1,000. Random-orientation steering performs no better than isotropic attention. PROMISE12 shows no gain.
Any one of these failing would narrow or refute the hypothesis. The most informative failure would be the vesselness channel tying, which would mean directional priors help small-sample vessel segmentation through inputs alone.
References
- A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery
- Analyzing Oriented Patterns
- Revisiting non-learned operators based deep learning for image classification: a lightweight directional-aware network
- Learn from orientation prior for radiograph super-resolution: Orientation operator transformer
- Tubular anisotropy for 2D vessel segmentation
- Construction of Compactly Supported Shearlet Frames
- Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks
- Stable Anisotropic Regularization
- Thrifty Neural Architecture Search for Medical Image Segmentation (Student Abstract)
- CDA-Net for Automatic Prostate Segmentation in MR Images
- FEDD - Fair, Efficient, and Diverse Diffusion-Based Lesion Segmentation and Malignancy Classification