Segment-Level Topology Loss on the Line Graph of Vessel Skeletons

Thin-structure segmentation models, such as those for retinal vessels or road networks, usually fail in a specific way. They trace the thick trunks well and then break the thinnest branches into disconnected pieces, often through gaps only one or two pixels long. Standard losses barely notice these gaps, because a missing pixel costs about the same whether it sits in the middle of a background region or cuts a capillary in half. We propose to train such models with a loss defined on the line graph of the ground-truth skeleton. A line graph is a re-drawing of a graph in which every original edge becomes a node and two such nodes are linked whenever their edges meet at a shared endpoint. For vessels, this turns each vessel segment into a unit that the loss scores once, and turns each pair of segments meeting at a junction into a connection that the loss also scores once.

The core idea is to change how the loss pools errors, not which pixels it looks at. On each segment, the loss takes a smooth minimum of the predicted probabilities along that segment's centerline, so that a single gap counts as losing the whole segment. Each segment then contributes equally regardless of how thick or long it is. Each pair of segments that should connect through a junction receives its own continuity penalty. Both the skeleton and its line graph are built once from the annotation before training, so the method keeps the cost of an ordinary differentiable loss while carrying graph-level structure that existing pixel-space topology losses lack.

Core Claim

Existing topology-aware losses average their signal over pixels. clDice builds a soft skeleton by repeated min- and max-pooling and measures what fraction of skeleton pixels fall inside the other mask. A one-pixel break in a 60-pixel capillary therefore costs under 2% of that capillary's term, and long thick vessels contribute far more skeleton pixels than short thin ones. We claim that pooling the topological signal per vessel segment, and per segment pair at each junction, will reduce fragmentation of 1–2 pixel micro-vessels more than any pixel-pooled loss.

The pooling is a smooth minimum along the segment, with segments weighted uniformly. Fragmentation of these micro-vessels is the failure that BTS-DSN reports persists even with bidirectional multi-scale connections. We expect the gain at little or no cost to overlap metrics.

The line graph is the right object for the junction term. In the primal skeleton graph, a junction is a single node of degree $k$, and asking whether that node is covered says nothing about whether every incoming branch actually links to every other. In the line graph, the same junction becomes the $k(k-1)/2$ edges between its incident segments, and each pair can be checked explicitly.

Method

Let $\tilde p_i$ be the predicted foreground probability at pixel $i$, max-pooled over a 3×3 window so that one-pixel centerline misalignment is tolerated. From the ground-truth mask we extract a skeleton and split it at branch points into segments $s$ with centerline pixel sets $P_s$. These segments form the node set $V_L$ of the line graph. Its edge set $E_L$ contains a pair $(s,t)$ whenever $s$ and $t$ share a junction. For each such pair, $Q_{st}$ is the path formed by the last few pixels of $s$, the junction pixels, and the first few pixels of $t$. The two new terms are

$$L_{\text{seg}} = \frac{1}{|V_L|}\sum_{s\in V_L}\Big(1-\operatorname{softmin}_\tau\{\tilde p_i : i\in P_s\}\Big),\qquad L_{\text{junc}} = \frac{1}{|E_L|}\sum_{(s,t)\in E_L}\Big(1-\operatorname{softmin}_\tau\{\tilde p_i : i\in Q_{st}\}\Big),$$

where $\operatorname{softmin}_\tau(x) = -\tau\log\frac{1}{n}\sum_j e^{-x_j/\tau}$ is a smooth minimum over the $n$ values with temperature $\tau$. As $\tau$ grows it approaches the mean, which recovers pixel-averaged centerline recall. This continuous knob doubles as a built-in ablation.

The full objective adds these terms to the standard compound loss. Binary cross-entropy supplies stable gradients early in training. Dice supplies size-invariant region overlap. The topology-precision half of clDice penalizes predicted skeleton lying outside the annotation, because both new terms only measure recall:

$$L = L_{\text{BCE}} + L_{\text{Dice}} + \lambda_p\,(1-T_{\text{prec}}) + \lambda_s L_{\text{seg}} + \lambda_j L_{\text{junc}}.$$

Two design choices follow from known weaknesses of the inputs.

First, the line graph is built only on the sparse ground-truth skeleton, offline. This sidesteps the memory blow-up that CensNet and Dual-Primal Graph Convolutional Networks report for dense graphs; vessel skeletons have junction degree around 3. It also avoids differentiable skeletonization of predictions inside the new terms. Those two papers supply the conceptual tool, the idea that edges deserve to be first-class units, but they apply it only to citation and molecule benchmarks. We use the line graph to define what the loss counts, not to run graph message passing.

Second, early in training every smooth minimum is near zero, so the gradients are uninformative. Following the logic of compound losses, where the distribution loss establishes localization before overlap terms take over, $\lambda_s$ and $\lambda_j$ are ramped in after the Dice overlap stabilizes. The ramp schedule is fixed in advance, not tuned per dataset.

Relation to Prior Work

Three lines of work motivate the gap. BTS-DSN, CE-Net and UNet++ address scale variance with architecture: short connections plus deep supervision, dilated and multi-kernel pooling context, and nested dense skips, respectively. All three still train with pixel losses, and BTS-DSN explicitly reports micro-vessel fragmentation. We change the loss and keep their architectures as backbones.

clDice supplies the closest method. We reuse its precision term but replace pixel-averaged sensitivity with segment- and junction-pooled terms.

DTU-Net adds a second refinement U-Net trained by triplet loss against synthetically broken masks. It shows that break-aware supervision helps, but it adds a second inference stage. Our method is a drop-in loss with single-stage inference.

Two works constrain the design. Towards Generalizable Retina Vessel Segmentation with Deformable Graph Priors uses a primal vessel graph as a deformable statistical prior inside a variational model. It argues for graph structure but operates on junction-centric primal graphs with heuristic offline construction, and it targets domain shift rather than fragmentation. Biologically-Constrained Graphs for Global Connectomics Reconstruction fixes split errors with lifted multicuts but only as non-differentiable post-processing. We place structure in training, where it can shape features.

Research Program

We will first build the preprocessing. This means skeletonizing the ground truth, splitting at branch points, pruning spurs shorter than three pixels, and constructing $E_L$. We will release segment-width statistics per dataset. We will then train the loss on two backbones: a plain U-Net and CE-Net, the strongest multi-scale grid architecture in the file. Using both tests whether the gain is additive to architectural multi-scale handling or redundant with it.

The analysis centers on width-stratified segment recall. A segment counts as recovered when every centerline pixel of its 3×3 neighborhood is predicted foreground. We report this in width bins of 1–2, 3–5 and more than 5 pixels, alongside junction recall: the fraction of line-graph edges whose path $Q_{st}$ is fully predicted.

Annotation noise is a known risk. A smooth minimum concentrates gradient on the weakest pixel, so a misplaced or gapped annotation becomes a strong wrong signal. We will test this directly using the synthetic vessel label erasing in the SGL codebase, introduced by Study Group Learning, and we will also use that codebase's training pipeline for DRIVE and CHASE_DB1.

Baselines

The naive reference point is the same backbone trained with BCE plus Dice at matched epochs and augmentation. It tells us how much fragmentation the compound loss alone leaves behind.

The strongest loss-level competitor is soft-clDice combined with soft Dice, following clDice. It is differentiable, cheap, and targets exactly the same failure. It will be run on the identical backbones with an equal search budget: the same number of trials over $\alpha$ and skeletonization iterations as we spend on $\lambda_s$, $\lambda_j$ and $\tau$, selected on the same validation split. No absolute DRIVE number for clDice is reported in the file. Beating it means higher width-stratified 1–2 pixel segment recall, higher junction recall and lower Betti error, with Dice within 0.5 points.

The strongest reported system numbers come from two architectures. BTS-DSN reports AUC 0.9806 and F1 0.8249 on DRIVE, and AUC 0.9859 and F1 0.8421 on STARE. DTU-Net reports IoU 73.86% and Betti error 0.8597 on DRIVE. We will rerun DTU-Net under our splits where possible, since its two-stage design is the most direct topology-focused competitor, and we will also quote the published figures. A single-stage CE-Net trained with our loss that matches DTU-Net's Betti error without a second network would be a clear win. Beating it outright would be stronger. Matching BTS-DSN's F1 shows we did not trade overlap for connectivity.

Three ablations isolate the key ingredient. In the first, we set $\tau$ large, which turns segment pooling into a segment mean; this isolates the minimum-versus-average question. In the second, we drop $L_{\text{junc}}$, which isolates the line-graph edges. In the third, we replace $L_{\text{junc}}$ with a primal junction-node term that covers only the junction pixels; this isolates whether the pairwise dual edges matter beyond node coverage.

Why not just…?

Why not just reweight clDice by inverse vessel radius? This is the most serious threat. It gives thin vessels equal weight without any graph construction, and if scale imbalance were the whole story it would match us. It does not, however, change the pooling: within a segment, a gap still costs a fraction proportional to its length. We include radius-reweighted clDice as an extra baseline under the same tuning budget. If it matches our full loss on 1–2 pixel segment recall, the conclusion is that the smooth minimum and junction terms are unnecessary, and we will report that.

Why not just use a stronger multi-scale architecture? CE-Net and UNet++ enlarge context without resolution loss. Yet BTS-DSN, which already combines bidirectional multi-scale paths with deep supervision, still reports fragmented micro-vessels. Training our loss on CE-Net directly tests whether architectural and loss-level fixes are complementary.

Why not use persistent-homology losses, which carry formal topological guarantees? DTU-Net reports TopoLoss needing more than 43 hours of training on its ultrasound data versus 17.2 hours for its own method. That is impractical for broad tuning. We will report wall-clock overhead relative to soft-clDice, which the file reports at under 10% overhead.

Why not fill gaps afterward with graph post-processing or a GNN over the predicted skeleton? Post-processing such as the lifted multicuts in Biologically-Constrained Graphs cannot send gradients back to the encoder, and it depends on the initial prediction containing the right candidates. A GNN over predicted skeletons inherits non-differentiable graph extraction and adds latency. We will compare against a simple morphological gap-closing step applied to the baseline. If closing recovers most segment recall at equal false-positive rate, our training-time loss is not justified.

Proposed Experiments

The main benchmarks are DRIVE, STARE and CHASE_DB1 for in-domain retinal segmentation, and Massachusetts Roads, where junction structure is dense and functionally critical. We add CHASE-to-HRF transfer to check that topology gains survive domain shift; GraphSeg reports a 7.43% Soft Dice margin over its next-best method in that setting. Metrics are F1/Dice, AUC, clDice, Betti error, width-stratified segment recall, junction recall and connected-component count error, averaged over three seeds with paired bootstrap intervals.

The decisive check is the 1–2 pixel segment-recall bin. The full loss must beat both standard and radius-reweighted clDice there. The large-$\tau$ ablation must lose most of that gain, and the primal-junction ablation must give lower junction recall than the line-graph term. If the gain appears only in thick-vessel bins, or vanishes at large $\tau$ without the junction term adding anything, the line-graph framing is not doing the work.

A secondary check is the label-erasing experiment. If segment recall improves on clean labels but Dice degrades sharply under erased labels relative to clDice, the smooth minimum is overfitting annotation gaps, and the tolerance window must widen before any clinical claim.