Segmentations That Keep Their Topology When Rotated

Topology-aware segmentation losses now produce vessel, road and neuron masks whose connectivity matches the ground truth. Connectivity here means the number of connected pieces and the number of enclosed holes. These guarantees, however, hold only in the frame where the mask was produced. Rotate that mask by an arbitrary angle on a pixel grid and round coordinates back to integers, as happens in data augmentation, registration, atlas alignment, or multi-view fusion, and one-pixel-wide diagonal vessels fall apart into fragments. Nearby branches can also fuse at a corner. Discrete geometry has known since the work of Ngo, Kenmochi, Passat and Talbot that this breakage has a purely local cause. If every pair of neighbouring pixels sits inside a full 2×2 block of the same value, a property they call regularity, then no rotation or translation can change the image's topology. Their theory was tested almost only on disks, half-planes and a single qualitative retinal crop. It has never met a modern segmentation network.

This note proposes to close that loop in two steps. The first is a measurement study. It asks, across real public datasets, how often ground-truth annotations and outputs from the best topology-aware segmenters change topology under arbitrary rigid motions. It also asks whether the cheap local regularity check predicts which masks will break. The second is a training method. It adds a differentiable, local penalty on the forbidden 2×2 configurations, so that a network learns to emit masks that are regular by construction. The goal is to obtain the rotation guarantee without the heavy-handed geometric distortion that post-hoc correction introduces. In two sentences: topology losses make masks correct at rest but leave them fragile under rotation, because they accept one-pixel-thin structures. We penalise exactly the local patterns that discrete topology proves cause the fragility, and we benchmark rotation stability on real data for the first time.

Core Claim

Regularity, a local property that can be checked in linear time, is the missing ingredient in topology-preserving segmentation. Our claim has three parts. First, on real datasets, a large share of both annotations and predictions from methods such as Topograph are non-regular and do change topology under random rigid motions. Second, the local certificate of Ngo et al. is a useful predictor of that breakage, even though it is only a sufficient condition. Third, a soft regularity penalty added to a topology loss can sharply reduce rotation-induced topology changes while keeping Dice and topology error in the original frame close to the base loss. If the third part fails, the first two still form a benchmark the field does not have.

Relation to Prior Work

Three groups of papers play three different roles.

The discrete-topology line supplies the mechanism and motivates the gap. Ngo et al. on regularity defines regularity for binary and label images and proves that it is sufficient for invariance under rigid motion. It also gives two repair procedures: homotopic pixel flipping and 2×2 up-sampling. Its DRIVE demonstration is visual only. Ngo et al. on sufficient conditions and the simple-point / transformation-graph work show that checking 5×5 neighbourhoods against a precomputed lookup table certifies invariance in linear time. Both are tested on geometric primitives. Kenmochi et al. gives the geometric reading of regularity as "every adjacent pair inside a 2×2 block", which is what makes it differentiable in our hands. Passat et al. extends homotopic transforms to affine maps through a cellular-complex optimisation, evaluated only on synthetic shapes and too slow for training loops. We reuse their definitions and lookup tables unchanged. What we add is real-data evaluation, and moving regularity from a post-hoc repair into a training signal.

The topology-loss line supplies the strongest competitors, and it constrains our design: we must not lose what these methods already achieve. clDice matches soft skeletons and lowers topology errors on DRIVE, Massachusetts Roads and VesSAP. Betti matching matches persistence features between prediction and label. Topograph penalises only "critical" misclassified regions and is the fastest of these. Its loss averages about 96 ms, against about 327 ms for Betti matching and about 603 ms for HuTopo, as reported in that paper. All three evaluate in a fixed frame and all three accept one-pixel-thin connections. Topograph's notion of a "regular" node concerns the error boundary between prediction and label, not the 2×2 thickness condition. This is precisely the gap our penalty targets.

Schneider et al. motivates the work from a different side. Continuous implicit shape models with guaranteed path-connectedness still disconnect once rasterised, because zero-width connections vanish on the grid. That is the same discretisation failure seen from the continuous side. It suggests that moving to continuous representations does not escape the problem, so the fix has to live on the grid.

Research Program

Benchmark. The first component is a rotation-stability benchmark on real 2D data. We take DRIVE, CHASE, HRF and STARE (retinal vessels), Massachusetts Roads (aerial road networks), and Cremi and Platelet (electron microscopy). For every ground-truth mask and every prediction, we sample rigid motions with angles drawn uniformly from the circle, explicitly excluding multiples of 90°, and with sub-pixel translations. We apply them with the Eulerian convention: each target pixel takes the nearest source value, the setting in which the theory holds. For each mask we record three quantities: whether its homotopy type changed, the changes in component and hole counts, and whether the 5×5 lookup-table certificate held beforehand. Comparing certificate verdicts with observed breakage gives an empirical answer to how conservative the sufficient condition is on real shapes. That is the open necessity question of Ngo et al., studied empirically.

We also report a second, more practical measure: equivariance consistency. Segmenting the rotated image should give the same topology as rotating the segmentation. This measure ties fragility to augmentation and registration pipelines.

Method. The second component is a soft regularity penalty. Let $p_i \in [0,1]$ be the predicted foreground probability at pixel $i$. For a 4-adjacent pair $(i,j)$, let $B_1, B_2$ be the two 2×2 blocks containing both pixels, and let $\pi_B$ be the product of the probabilities of the block's other two pixels. The foreground violation term is

$$ R_{\text{fg}} = \sum_{(i,j)} p_i\, p_j \,\Bigl(1 - \max_{B \in \{B_1,B_2\}} \pi_B\Bigr), $$

which is near zero exactly when every confident foreground pair lies inside a confident foreground block. $R_{\text{bg}}$ is the same expression with $1-p$ in place of $p$; it keeps thin background gaps between neighbouring branches from closing. The total penalty is $R = R_{\text{fg}} + R_{\text{bg}}$. It is a fixed set of shifted products, so it costs one convolution-like pass on the GPU. It is added with a weight $\lambda$ to a base topology loss, primarily Topograph. Because regularity is sufficient for invariance, a thresholded output that passes the lookup table carries a hard guarantee under all rigid motions. Outputs that do not pass can be finished with a small number of homotopic flips, and we report how many flips were needed.

The real cost. True capillaries and thin roads in DRIVE and Roads are often one pixel wide, so regularity forces some thickening. We will quantify this trade-off directly as a function of $\lambda$. The trade-off is plotted as rotation-induced topology change against Dice and Hausdorff distance in the original frame. This makes the geometric-distortion concern raised by Kenmochi et al. a measured quantity rather than a caveat.

Baselines

Do-nothing baseline. The naive reference is a U-Net trained with Dice. Its thresholded mask is rotated with nearest-neighbour rounding. It sets the floor for how often topology changes under motion.

Strongest competitor. The strongest existing method is Topograph, run with the authors' loss on the same backbone, the same splits and the same augmentation. It gets the same hyperparameter search budget as our method: an equal number of trials over learning rate and loss weight. clDice and Betti matching are included under identical conditions. Topograph is the strongest because it is both the most accurate in the original frame across the datasets it reports and the cheapest. Any regularity method must therefore beat it on rotation stability without falling behind on its own metrics.

No reported result was found on Lacuna for rotation-induced topology change of any of these methods on any dataset. No reported numeric Betti error on DRIVE for clDice appears in the file either. The only reported Topograph figures in the file are runtimes, around 96 ms per loss evaluation, plus the qualitative claim that it matches or beats the other topology losses on DIU and Betti matching error. So we will re-run all baselines ourselves rather than copy numbers.

What counts as beating Topograph. Beating it means three things together. Rotation-induced homotopy changes must be substantially lower, judged with paired bootstrap confidence intervals over test images and sampled motions. Betti matching error and DIU in the original frame must not be significantly worse. The Dice loss must be bounded, and we will report it whatever it turns out to be.

Post-hoc regularisation. The decisive comparison is the homotopic regularisation of Ngo et al. applied after the fact to Topograph outputs. Up-sampling regularisation from the same paper is also included, with resolution restored afterwards.

Ablations. The ablations isolate the key ingredient in three ways. The first applies the penalty with plain Dice and no topology loss. The second uses the foreground-only term. The third uses a crude stand-in, a minimum-thickness penalty implemented with soft erosion, to test whether the specific forbidden-pattern structure matters beyond "be thicker".

Why not just…?

Why not just regularise the predictions afterwards with Ngo's homotopic flipping? This is the real threat. It already gives the guarantee, and it is cheap to run. Our argument against it is that it preserves the homotopy type of the prediction, including that prediction's mistakes. It also thickens structures blindly, without image evidence, whereas a trained network can place the extra pixels where the image supports them. That is an argument, not evidence. The comparison against "Topograph plus post-hoc flipping" will settle it. If post-hoc flipping matches us on rotation stability and on Hausdorff distance and Dice, we will say so. The contribution then reduces to the benchmark and to the finding that post-hoc regularisation is enough. That result would itself be new, since the procedure has never been quantitatively evaluated on real data.

Why not just rotate the soft probability map with bilinear interpolation and threshold afterwards? This is what many real pipelines actually do, and it may avoid some breakage. Bilinear interpolation of a one-pixel diagonal structure, however, dilutes values toward 0.5 exactly at the sub-pixel offsets where rounding fails. So we expect fractures to persist, but we do not assume it. This variant becomes a benchmark condition for every method. If it eliminates fractures on its own, the problem is narrower than claimed, and the benchmark will show it.

Why not just train with heavy random-rotation augmentation? Augmentation teaches the network to segment rotated images. It does not change the fact that the resulting thin mask breaks when the mask is rotated. Pixel-wise losses also do not penalise disconnection when overlap stays high, as clDice and Topograph argue. All of our baselines already use rotation augmentation, so the comparison directly measures what augmentation leaves on the table.

Why not just up-sample or apply morphological closing? Up-sampling regularises natively, but it multiplies memory, and its guarantee is lost when the mask is brought back to the original resolution (Ngo et al.). Closing is not constrained by topology and merges nearby branches. Both are run as baselines, and we report whether their gain in stability costs false merges, counted as drops in the number of components and holes.

Proposed Experiments

Experiment 1: annotation fragility. We run the benchmark on ground-truth annotations alone. If a large fraction of expert DRIVE or Cremi masks change topology under random rotation, then any registration-based evaluation of topology is itself unreliable. That is a finding about the field's evaluation practice, independent of our method.

Experiment 2: predictive value of the certificate. We measure the certificate's false-alarm rate: how often a mask that fails the 5×5 check nonetheless survives every sampled motion. A high rate would mean the sufficient condition is far from necessary on real shapes and would motivate weaker, angle-range-specific conditions. A low rate would justify using the linear-time check as a training-free monitor.

Experiment 3: training comparison. We train each method on every dataset with three seeds and report the following in both the original frame and under rotation: Dice, clDice, Betti matching error, DIU, Hausdorff distance, homotopy-change rate and equivariance consistency. Cross-dataset retinal transfer, trained on CHASE and tested on HRF following the setting of GraphSeg, checks whether regular outputs stay regular under domain shift.

Experiment 4: trade-off curve. Sweeping $\lambda$ gives a trade-off curve between rotation stability and geometric accuracy. A proposal-killing result would be any of the following: the stability gain disappears once bilinear soft-map rotation is used; post-hoc flipping matches the trained penalty at equal geometric error; or the Dice cost on thin-vessel datasets exceeds the gap between Topograph and plain Dice. We will report any of these plainly.