Teaching Skip Connections When to Stay Quiet: Occlusion-Supervised Gating for Thin-Structure Segmentation from Few Scans

Segmentation networks for thin anatomy, such as retinal vessels, coronary arteries and neuron membranes, usually follow the U-Net design. An encoder compresses the image into coarse, context-rich features, a decoder expands them back to a pixel map, and "skip connections" copy high-resolution encoder features straight across to the decoder so that fine detail survives. That design works when the structure is visible. It works against the model when a stretch of vessel is hidden by a lesion, blurred by motion, or indistinguishable from surrounding tissue. At those pixels the skip connection faithfully delivers crisp local evidence that says "no vessel here," and that evidence can overrule the deep features, which have enough context to guess that the vessel continues. The result is the familiar broken, fragmented vessel map. We propose to paste realistic synthetic dropouts directly onto the labeled vessels of a small training set while keeping the full, unbroken label. Because we placed each occluder ourselves, we also know exactly where the local evidence is untrustworthy. We use that location as a free, annotation-free training target for a small gate on every skip connection, which learns to close where the input is corrupted and stay open everywhere else.

The opportunity is sharpest when training data is tiny: 20 to 40 images or volumes, the norm in clinical work. In that regime an extra, exact supervision signal costs nothing to produce. Occlusion-driven shape learning has so far been shown only with hundreds of thousands of synthetic images. The question is whether it transfers to this setting without teaching the model to invent vessels that are not there.

Core Claim

In encoder-decoder segmentation of thin structures, identity skip connections actively suppress the recovery of hidden segments, which we call "amodal completion": predicting the full shape of a structure including its hidden parts. Two ingredients fix this under small-sample constraints. The first is synthetic dropouts anchored to the vessel's centerline, trained against the complete label. The second is per-pixel skip gates supervised with the known dropout mask on occluded samples and held open on clean samples. The claim is falsifiable at three levels. Centerline-anchored dropout with full labels should beat random erasing on gap recovery. Supervised gates should add recovery inside occluded gaps beyond what the same augmentation gives a plain or attention-gated U-Net. Neither should raise the rate of spurious bridges on clean images.

Concretely, at skip scale $l$ with encoder features $e_l$, the decoder receives $g_l \odot e_l$. Here $g_l \in [0,1]^{H_l \times W_l}$ is predicted by a lightweight head from $e_l$ and the upsampled decoder state. The gate is trained with

$$ \mathcal{L} = \mathcal{L}_{seg}(\hat{y}, y) + \lambda \sum_l \mathrm{BCE}\big(g_l,\; 1 - \mathrm{down}_l(m)\big), $$

where $y$ is the complete (amodal) vessel label and $\hat{y}$ the prediction. The synthetic occlusion mask $m$ equals 1 on pasted-over pixels and is all zeros on clean samples, so the target is "fully open" there. $\mathrm{down}_l$ resizes $m$ to scale $l$, and $\lambda$ weights the gate term. The clean-sample term plays the role that dual clean/occluded supervision plays in Segment Anything, Even Occluded: it discourages the model from completing structures that are not there.

Where This Sits

Three strands of prior work motivate the idea without testing it. The first is synthetic occlusion for amodal learning. Amodal Instance Segmentation showed that pasting occluders onto fully visible objects forces networks to infer hidden extent. BCNet in Occlusion-Aware Instance Segmentation Via BiLayer Network Architectures separates occluder and occludee into two layers in the mask head and trains on a 100,000-image synthetic set. SAMEO in Segment Anything, Even Occluded adapts EfficientSAM with 300,000 synthetic Amodal-LVIS images. We borrow the paired-label idea from these papers but change three things. We work on thin anatomy, not objects. We use tens of samples, not hundreds of thousands. We intervene at the multi-scale skip connections, which BCNet leaves as static identity passes. SAMEO also constrains our design: it reports that samples under roughly 10% visibility teach little and samples above 90% carry almost no amodal signal. We therefore control dropout length relative to vessel caliber and local context rather than erasing uniformly.

The second strand is learned skip connections. Toward fast and accurate human pose estimation via soft-gated skip connections learns a per-channel scalar gate and finds most gates collapse near zero, which is evidence that raw skips carry redundant signal. UDTransNet in Narrowing the semantic gaps in U-Net with learnable skip connections recalibrates encoder features with decoder-guided attention. MM-UNet in MM-UNet: Morph Mamba U-shaped Convolutional Networks for Retinal Vessel Segmentation adds uncertainty-focused guidance in the skips and deformable kernels that follow tubes. All of these gates or attention maps are learned only indirectly from the segmentation loss, and all are content-dependent in ways never tested against occlusion. Our gate is spatial, not per-channel, and it receives a direct target that says where local evidence is corrupt. That target is only available because the occlusions are synthetic.

The third strand is small-sample thin-structure segmentation. The Importance of Skip Connections in Biomedical Image Segmentation reached strong membrane continuity on the 30-slice EM ISBI 2012 set with long plus short skips and standard warping augmentation. NA-UNETR in A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery handles 20 non-contrast CT volumes by pretraining on ImageCAS and adapting with LoRA, a method that trains small low-rank weight updates. That paper states that performance is limited by the visibility of the anatomy, and it treats dropouts only through the loss. Unveiling COVID-19 from Chest X-ray with deep learning is a warning: on small cohorts, deep models latch onto artifacts. A synthetic pasting boundary is exactly such an artifact, so our occluder generator must leave no detectable seam, and we test for this explicitly.

Research Program

The first component is a centerline-anchored dropout generator for 2D fundus images, 2D electron microscopy, and 3D CT. From the training label we extract the skeleton and sample segments whose length is a multiple of local vessel width. We overwrite each segment with one of four corruptions. The first is texture replacement: we copy a patch of nearby non-vessel tissue and blend it with a feathered edge so that no sharp seam survives. The second is partial intensity fade toward background, which mimics isointense tissue in non-contrast CT. The third is directional blur, which mimics motion. The fourth is a lesion-like blob sampled from real lesion appearance where available. The label stays complete throughout. We also produce the corruption mask $m$. A seam detector, a small classifier trained to tell real patches from pasted ones, audits whether the synthetic boundaries are detectable. If they are, we strengthen the blending before any segmentation experiment.

The second component is the gated skip module. It adds well under 1% parameters to a U-Net or to the NA-UNETR backbone and uses the loss above. We compare three training schemes: gate supervision always on, gate supervision annealed to zero, and gates learned with no gate loss. This separates the value of the direct target from the value of merely having gates.

The third component is an analysis of skip dominance. On models trained without gates, we zero each skip scale at test time inside synthetic gaps and measure how much gap recovery changes. If removing the finest skips improves recovery inside gaps, that directly confirms the mechanism the proposal rests on. If it does not, the gating idea loses its rationale, and we would report the augmentation result alone.

Baselines

The naive reference is a standard U-Net with the geometric augmentation used in The Importance of Skip Connections in Biomedical Image Segmentation: flips, rotations and spline warping. It is trained on identical splits with the same epochs and learning-rate search. Beating it only on overall F1 would be weak evidence. The result that matters is higher gap recovery with no rise in false bridges.

The strongest vessel-specific competitor is MM-UNet. It reports 89.59% F1 on DRIVE and 91.77% F1 on STARE (MM-UNet), and its skip guidance explicitly targets thin-branch continuity. We will retrain it on our splits with its published augmentation and give it the same hyperparameter budget as our method. We then add our gates and augmentation on top of it. Beating it means matching its overall F1 within noise while improving gap recovery on real lesion-occluded vessels and on held-out synthetic gaps.

For EM, the reference is the long-plus-short-skip network, reported at $V_{rand}=0.969$ and $V_{info}=0.986$ on ISBI 2012 (Drozdzal et al.). For 3D coronary arteries, the reference is NA-UNETR with ImageCAS pretraining and LoRA. It reports 45.64% Dice and 38.16 mm HD95 on LAD-SEG against 42.54% Dice for nnU-Net (NA-UNETR). LAD-SEG is institutional, so if it is unavailable we reproduce the regime by adapting from a disjoint ImageCAS pool to a 20-volume ImageCAS subset. That subset is degraded by contrast reduction to mimic non-contrast CT, and we evaluate on held-out volumes. No reported result was found on Lacuna for this constructed setting, so all numbers there are our own reproductions.

Two ablations isolate the key ingredient. The first is "augmentation only": our dropout generator and full labels, with identity skips. The second is "augmentation plus unsupervised gates": the same augmentation with UDTransNet-style recalibrated skips (UDTransNet) or learned soft gates in the style of Bulat et al.. Supervised gating must beat both on gap recovery to justify itself.

Why not just…?

Why not just use random erasing (CutOut) but keep the full label? This is the most serious threat, because it already supplies amodal supervision with trivial code. Our reason to expect it to fall short is geometric. Uniform rectangles rarely land on a thin vessel. When they do, they erase context on both sides and produce sharp seams, which is the kind of shortcut Tartaglione et al. show small models exploit. We do not argue the point. We run full-label CutOut as a baseline with occlusion area matched to our generator, both with and without our gate. If matched-area CutOut recovers gaps as well as centerline-anchored dropout does, the generator is not a contribution and we will say so.

Why not just rely on learned attention in the skips, which should discover occlusions implicitly? UDTransNet and soft gating do reweight skip features. However, the collapse of Bulat et al.'s gates toward zero suggests that indirect training learns where skips are redundant, not where they are wrong. With 20 to 40 training images there is little signal to learn the latter. The "augmentation plus unsupervised gates" ablation settles this directly. If it closes the gap to supervised gating, the direct gate target adds nothing.

Why not just repair breaks afterwards with snakes, conditional random fields or watershed? The left-ventricle pipeline in A Left Ventricular Segmentation Method on 3D Echocardiography Using Deep Learning and Snake shows that contour models need a good initialization and snag on noise at edge dropouts. Drozdzal et al. matched post-processed pipelines without any post-processing. Post-processing also cannot distinguish a real vessel ending from an occluded one. We nonetheless include a centerline-gap-closing post-processing step on the naive U-Net as a comparison, scored on the same false-bridge metric, because a method that bridges more gaps while also bridging real endings has not won.

Why not just pretrain big and adapt, as NA-UNETR does? Pretraining and our method are complementary. NA-UNETR remains limited by visibility after pretraining on 1,000 volumes, and SAMEO shows that even foundation models predict only visible pixels unless they are trained on occlusion pairs. We therefore apply our method on top of the pretrained and LoRA-adapted backbone rather than instead of it.

Proposed Experiments and Decisive Checks

We train on DRIVE (40 images), STARE (20 images), EM ISBI 2012 (30 slices) and the 20-volume coronary setting. For each, we also run learning curves at 25%, 50% and 100% of the training set, with five seeds per condition. Standard metrics are F1 and sensitivity for retina, $V_{rand}$ and $V_{info}$ for EM, and Dice and HD95 for CT.

Three purpose-built measurements carry the claim. Gap recovery is the fraction of ground-truth centerline recovered inside occluded regions. We measure it on held-out images with synthetic dropouts of each corruption type. For real occlusions we use pathological STARE images, scoring vessel pixels adjacent to lesions. The false-bridge rate is the number of predicted connections between components that are separate in the label, measured on clean test images and near true vessel endpoints. The connected-component count relative to ground truth measures fragmentation.

The decisive checks are these. First, the skip-zeroing analysis must show that fine skips hurt recovery inside gaps in ungated models. Second, supervised gating must beat both augmentation-only and unsupervised-gate ablations on gap recovery beyond seed variance, with no increase in false bridges. Third, predicted gate maps on real lesion-occluded STARE vessels must close over the lesions without any lesion ever being synthesized for the gate, which tests whether the gate transfers from synthetic to real corruption. Fourth, the seam detector must fail to separate synthetic from real patches, and the gain must survive swapping the generator's blending method. If the first check fails, we drop the gating component and report only the augmentation study. If the third fails, the method is a synthetic-benchmark trick, and we will present it that way.