Keeping Thin Structures Alive Inside the Encoder

Segmentation networks routinely break thin things: retinal capillaries, road networks, concrete cracks and nerve canals come out fragmented, even when the overall pixel overlap looks good. The usual fixes act on the network's output. They add losses that reward an unbroken centerline, or they redesign the decoder that upsamples features back to full resolution. A separate line of work tries to stop downsampling from discarding detail in the first place, but it has been tested only on classification and small-object detection. This proposal asks a question that sits between the two lines and has not been measured: at which depth of the encoder does a thin structure stop being represented, and can we keep it there by telling each encoder stage what the structure's skeleton should look like at that stage's resolution?

The concrete idea is to supervise every encoder stage with a skeleton target. A skeleton here is the one-pixel-wide centerline of the structure. Each target is computed offline from the ground truth by a downsampling procedure that is guaranteed not to change the structure's connectivity. An existing integer-programming method already does this kind of downsampling for binary masks, but it is not differentiable and so was never usable inside a network. We sidestep that limitation by using it only to make training targets. The network then gets a cheap recall-style loss at every scale, and we test whether this keeps connectivity information alive deep in the encoder better than swapping in lossless downsampling layers or adding topology losses at the output.

Core Claim

Our hypothesis is about where the damage happens. U-Net-style skip connections carry full-resolution pixels to the decoder, so the 1-pixel detail itself is not lost. What is lost is the deep features' awareness that a filament runs through a region, and the decoder needs that awareness to decide whether two nearby fragments belong to one vessel. Output-level losses cannot repair this. Skeleton Recall Loss computes recall against a precomputed, slightly thickened ("tubed") skeleton, and clDice computes a differentiable soft skeleton of the prediction. Both act only on the final map. We claim that adding stride-matched skeleton supervision at intermediate stages, with targets whose connectivity is exact, reduces Betti-number errors more than output-only supervision. A Betti number counts topological features: β0 counts connected pieces and β1 counts loops. We further claim this supervision is a cheaper and more effective intervention than replacing strided operations with SPD-Conv. SPD-Conv rearranges pixels into channels so that nothing is thrown away, but it multiplies channel count by the square of the downsampling factor.

With $s$ indexing encoder stages, the training objective is

$$\mathcal{L} = \mathcal{L}_{\text{Dice+CE}} + \sum_{s} \lambda_s \left(1 - \frac{\sum P_s \odot T_s}{\sum T_s}\right).$$

Here $P_s$ is the sigmoid output of a small auxiliary head on stage-$s$ features. $T_s$ is the tubed skeleton obtained by topology-preserving downsampling of the ground truth to stage $s$'s resolution. $\lambda_s$ is a per-stage weight. The term in parentheses is the soft skeleton-recall penalty of Skeleton Recall Loss, applied at every scale rather than only at the output.

Research Program

The first component is a diagnostic. On frozen encoders trained with ordinary Dice and cross-entropy, we fit a linear 1×1 probe at each stage to predict $T_s$. We report skeleton recall broken down by true vessel width (1, 2, 3–5, and more than 5 pixels) and by local orientation. This tells us at which stride and for which geometries thin-structure information disappears. It also checks the reasoning in the research file that axis-aligned Strip Pooling, which pools along full rows and columns, should dilute diagonal filaments. No paper in the file measures any of this.

The second component is target construction. Topology-Preserving Downsampling of Binary Images solves an integer program with connectivity and "no-island" constraints. It reports 0.0 Betti error on CNCB lung masks, against 0.04–0.96 for bicubic or pooling downsampling. Its authors concede that single-pixel filaments are sometimes sacrificed to their similarity objective. That failure mode is exactly the one we care about, so we add a constraint that every coarse cell containing a skeleton pixel stays foreground. We then measure how often the program becomes infeasible or slow at strides 2 through 16. The method is formulated for 2D. For 3D data we fall back to block-max-pooled skeletons and report this as a limitation.

The third component crosses the supervision scheme with the downsampling operator. The operators are strided convolution (as in NPNet), max pooling, SPD-Conv, and LIP, which uses learned importance weights so that pooling favors salient pixels. This settles empirically whether lossless downsampling and scale-wise supervision are substitutes or complements. It also resolves, on dense thin-structure data, the unresolved disagreement between SPD-Conv and NPNet about whether strided convolution is itself lossy.

Baselines

The naive reference point is a standard U-Net trained with Dice plus cross-entropy and no topology term. On DRIVE, Dynamic Snake Convolution reports this baseline at 80.73% Dice.

The strongest output-level competitor is the same U-Net trained with Skeleton Recall Loss. It is the cheapest published connectivity loss: about 8% extra training time and about 2% extra VRAM, and it runs on 13-class 3D TopCoW where clDice runs out of memory on a 40GB A100. It is the most direct rival because our method generalizes it. We found no reported Dice or Betti result on Lacuna for Skeleton Recall Loss on DRIVE, Roads, Cracks, ToothFairy or TopCoW, so we will rerun it. We also include clDice on the 2D datasets. Again, we found no numerical result on Lacuna for clDice on these benchmarks.

The strongest architectural competitor is DSCNet from Dynamic Snake Convolution. Its convolution kernels deform along a chain that follows the vessel, and it trains with a persistent-homology loss. It reports 82.06% Dice on DRIVE and up to 6.00% better overlap-until-first-error on cardiac CT angiography.

The resolution-preserving competitor is a U-Net whose strided layers are all replaced by SPD-Conv and trained with output-level Skeleton Recall Loss. No segmentation result exists for this combination on Lacuna.

All methods use the same nnU-Net-style preprocessing, augmentation and data splits. Each gets the same number of training epochs and the same hyperparameter search budget, which for our method covers the $\lambda_s$ values. Every method runs over three seeds, and we report peak memory and wall-clock time for each.

We count our method as beating a baseline only if it reduces β0 and β1 errors beyond seed variance on at least three of the five 2D/3D datasets, while matching that baseline's Dice within 0.5 points. Exceeding DSCNet's 82.06% DRIVE Dice is a secondary target.

The key ablation replaces the integer-programming targets with block-max-pooled skeletons. Max pooling keeps 8-connected curves connected, but it can merge parallel vessels and fill loops. This ablation isolates whether exact-topology targets matter or whether any multi-scale skeleton signal is enough.

Why not just…?

Why not just use Skeleton Recall Loss at the output? Skip connections may let the decoder recover everything, in which case intermediate supervision adds nothing. This is the main threat. The probing diagnostic and the head-to-head comparison settle it directly. If the probes show that deep stages already retain skeleton information under output-only training, and Betti errors match, we will report that output supervision suffices.

Why not just replace strided layers with SPD-Conv? Lossless rearrangement keeps the pixels, but nothing forces the network to use them for connectivity. SPD-Conv was validated only on detection AP and classification; it reports +13.15% small-object AP for YOLOv5-SPD-n on COCO. Its channel growth is also a known cost at segmentation resolutions. The operator-by-supervision grid shows whether SPD-Conv alone matches our supervision. If it does, at our memory budget, the simpler answer wins.

Why not just max-pool the ground-truth skeleton for deep supervision? It may well be good enough. Hole-filling and vessel-merging matter mainly in dense, looping networks such as roads and the Circle of Willis. The ablation above tests exactly this. We expect any gap to show up in β1 on Massachusetts Roads and TopCoW, not on sparse cracks.

Why not just train at higher input resolution? Upsampling the input shifts the stride at which filaments vanish without removing it, and it multiplies compute. As a resolution control, we train the plain-loss baseline at 2× input on the 2D datasets.

Proposed Experiments

The 2D experiments use DRIVE, Massachusetts Roads and the concrete cracks dataset. The 3D experiments use ToothFairy and TopCoW, both binary and 13-class. Metrics are Dice, clDice, β0 and β1 errors, and the stage-wise probe recall broken down by width. CMRxMotion serves as a stress test, since it reports left-ventricle Dice falling by more than 3 points and LVEDV error more than doubling under severe motion (CMRxMotion). There we ask whether multi-scale skeleton supervision of the thin myocardial wall reduces that degradation.

Three checks are decisive. First, the probes must show a width-dependent collapse at some stride under plain training, and our loss must remove it. If no collapse exists, the premise fails. Second, our loss must beat output-only Skeleton Recall on Betti errors at equal Dice. Third, integer-programming targets must beat max-pooled targets on loop-rich datasets. If they do not, the contribution reduces to multi-scale skeleton recall, and we will say so.