Learning Vessel-Continuity Energies Without the Normalizer

Segmentation networks fail in a predictable way when the target barely differs from its surroundings. A coronary artery in a CT scan taken without contrast agent, or a flat polyp against the colon wall, comes out fragmented, shrunk into blobs, or with an arbitrary edge somewhere in a gradual intensity ramp. The usual fixes are extra boundary branches, reverse attention and contour losses. They sharpen edges but still output one deterministic mask. They also never encode the property a clinician actually relies on, which is that a vessel continues smoothly from one cross-section to the next. That property involves several voxels at once, so it is a higher-order dependency (one that scores groups of three or more labels jointly, rather than single pixels or pairs). A probability model over whole masks that builds in such dependencies would let us sample plausible alternative masks where the image is ambiguous. Such models have been avoided because their partition function, the sum of unnormalized scores over every possible mask, is #P-hard to compute (Rong Ge et al.).

This note proposes a way around that obstacle. If the energy of a mask is a sum of small local clique terms, then the probability ratio between a mask and a slightly edited mask depends only on the cliques the edit touches, and the normalizer cancels. Discrete score matching trains a model using exactly such ratios. The new ingredient is the set of edits. Single-voxel flips teach almost nothing about continuity. Edits that act on geometry do: shifting a boundary run along its normal, extending or retracting a vessel segment, opening or closing a gap. We train a conditional energy-based model of vessel masks on these geometric edits and test whether it yields more connected and better calibrated segmentations of low-contrast tubular structures than strong deterministic networks.

Core Claim

A conditional energy over segmentation masks, made of learned local higher-order clique potentials, can be trained without ever computing or sampling for its normalizer. The training signal is a denoising discrete score matching objective whose corruption process consists of structured geometric edits rather than bit flips. We claim two things. First, the edit set is what makes higher-order potentials learnable: the same energy trained on single-voxel flips, or by pseudo-likelihood, will learn weaker continuity structure. Second, on vessels with degraded contrast, the resulting model improves topological continuity and the calibration of uncertainty in the boundary band, even where Dice barely moves.

How this differs from the closest work

The pieces exist separately, and each existing piece stops short in a different place.

Some prior work supplies the motivation. Higher-order vision energies showed that patch and curvature cliques preserve fine structures that pairwise models lose. Minimizing sparse higher order energy functions of discrete variables used sparse pattern dictionaries over 5×5 to 15×15 patches and cut Brodatz texture restoration error from 16.4% with a pairwise model to 10.5%. Curvature regularity for region-based image segmentation and inpainting scored pairs of adjacent boundary segments and solved a linear-programming relaxation, which kept thin appendages that length penalties snip off. Higher Order Energies for Image Segmentation minimized large-clique energies by Taylor expansion and QPBO. All three hand-design their potentials and return a single most-likely mask. They do not learn the potentials from data and give no probabilities. On the deep-learning side, Learning Active Contour Models for Medical Image Segmentation used the Chan-Vese energy as a training loss, but its length term is first-order total variation and it never defines a distribution over masks.

Other prior work constrains or supplies the method. Learning structured densities via infinite dimensional exponential families trained graphical models by score matching to avoid the normalizer. It stayed pairwise because kernel operators explode at higher order. That is a negative result we respect: we use small neural clique potentials, not kernel operators. Concrete Score Matching supplies the method. It generalizes score matching to discrete data through probability ratios over a neighborhood graph and proves those ratios identify the distribution when the graph is connected. Its neighborhoods are Hamming-1 flips, and it reports high gradient variance on non-grid neighborhood structures. Our contribution is to redefine that neighborhood graph as geometric edits on masks, and to accept that this may raise exactly the variance problem the authors warn about. The variance therefore has to be measured, not assumed away. Efficient Second-Order TreeCRF for Neural Dependency Parsing is the parsing analogue. It found that strong neural encoders still gain from explicit higher-order scoring. It could compute its normalizer exactly by dynamic programming, which is impossible on a 3D voxel grid; that is the case we target.

The Model

Let $x$ be an image volume and $y \in \{0,1\}^N$ a mask over $N$ voxels. The energy is

$$E_\theta(y, x) = \sum_{i} u_\theta(y_i; x) + \sum_{c} \phi_\theta(y_c; x),$$

The unary term $u_\theta$ is the per-voxel output of a standard 3D segmentation backbone. Each $c$ is an overlapping 3×3×3 clique of voxels, and $\phi_\theta$ is a small network that reads the clique's label pattern $y_c$ together with local image features. The probability of a mask is $p_\theta(y \mid x) = \exp(-E_\theta(y,x))/Z_\theta(x)$, where $Z_\theta(x)$ is the intractable sum over all masks.

For any edit that turns $y$ into $y'$,

$$\log \frac{p_\theta(y' \mid x)}{p_\theta(y \mid x)} = \sum_{c \in \mathcal{C}(y,y')} \big[\phi_\theta(y_c; x) - \phi_\theta(y'_c; x)\big] + \text{(touched unary terms)},$$

where $\mathcal{C}(y,y')$ is the set of cliques containing a changed voxel. $Z_\theta(x)$ cancels, and the cost of evaluating the ratio scales with the size of the edit, not the size of the volume.

Training uses the denoising variant of Concrete Score Matching. Ground-truth masks are corrupted by random structured edits drawn from a kernel $q(y' \mid y)$, and the model's ratios are fit to the ratios implied by that corruption. The edit kernel mixes four kinds of moves:

  • single-voxel flips;
  • shifts of a connected boundary run by one voxel along its normal;
  • extension or retraction of a vessel segment by one cross-section along a skeleton estimated from the current mask;
  • insertion or removal of a one-cross-section gap.

Single flips are kept so that the neighborhood graph stays connected, which is the condition under which the identifiability guarantee of Concrete Score Matching holds.

At test time we start from the backbone's thresholded mask and run a short Metropolis chain over the same edit set. This gives an approximate MAP mask and a set of samples, and voxel marginals and connectivity statistics are read off the samples. Sampling is used only at inference, never inside the training loop.

Research Program

The work has three stages.

The first stage builds the energy, the edit kernel and the ratio computation as batched GPU operations. It also measures gradient variance under each edit family, since this is the failure mode Concrete Score Matching flags for non-grid neighborhoods. If variance is prohibitive, we fall back to control variates computed from the single-flip terms.

The second stage trains on a controlled low-contrast regime. We take ImageCAS, the public set of 1,000 contrast-enhanced coronary CT angiography volumes, and synthetically compress the intensity gap between vessel and surrounding tissue while adding correlated noise, sweeping over several severity levels. This keeps clean labels while letting us dial the ambiguity up and down. The third stage transfers to real ambiguity: the LAD-SEG non-contrast planning CT set if access can be arranged, and the 2D polyp benchmarks. The polyp benchmarks test whether the method helps blob-like targets or only tubular ones.

Baselines

The naive reference is the backbone alone, a 3D U-Net of the nnU-Net type trained with Dice plus cross-entropy and thresholded at 0.5. It is exactly our unary term with no clique potentials. Every model shares its architecture, data splits, augmentation and a fixed GPU-hour budget, and every model gets the same number of hyperparameter trials.

The strongest existing method for low-contrast vessels is NA-UNETR from A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery. It restricts attention to local windows to preserve thin tubular geometry and reports 45.64% Dice and 38.16 mm 95th-percentile Hausdorff distance on LAD-SEG, 3.1 points above nnU-Net. On contrast-enhanced ImageCAS it reports 79.49% Dice. We will rerun it under our budget on the degraded-contrast ImageCAS sweep and also place our clique energy on top of it, because the question is whether the energy adds to the strongest backbone, not whether it replaces it. Beating it means improving centerline continuity, measured as the fraction of ground-truth centerline length lying in a single connected predicted component, and improving boundary-band calibration error at moderate-to-severe degradation, without losing more than one Dice point. Dice parity together with clear topology gains counts as a win.

The loss-based competitor is the active contour loss of Learning Active Contour Models for Medical Image Segmentation, which reports Hausdorff distance on the left ventricle dropping from 5.43 mm with cross-entropy to 4.73 mm with its loss on ACDC. We train it on the same backbone.

For uncertainty, the reference is a five-member deep ensemble of the backbone. No reported result was found on Lacuna for ensembles on these benchmarks, so we will produce our own numbers.

Three ablations isolate the key ingredients. The first keeps everything but restricts cliques to pairs, which tests whether higher order matters. The second trains the full energy with single-flip neighborhoods only, which is close to pseudo-likelihood and ratio matching and tests whether the structured edits matter. The third trains the same energy by persistent contrastive divergence with the same sampler, which tests whether score matching is needed at all.

For polyps, the reference number is CaraNet's 74.7% mean Dice on ETIS-LaribPolypDB from CaraNet, with MISNet's 0.903 and 0.918 mean Dice on Kvasir-SEG and CVC-ClinicDB from Multi-scale Information Sharing and Selection Network with Boundary Attention.

Why not just…?

Why not just use a strong backbone with a contour or boundary loss? For Dice this may well be enough, and we expect Dice gains to be small. The case against it rests on two other outputs. A deterministic mask cannot say where in an intensity ramp the edge plausibly lies. And a pixel-wise or first-order loss cannot penalize a one-voxel gap more than a one-voxel bulge, even though the gap breaks the vessel's topology. The NA-UNETR authors still report 38.16 mm HD95 on non-contrast LAD, which suggests large disconnected errors remain. The "backbone plus AC loss" and "NA-UNETR alone" arms test this directly. If they match our continuity and calibration numbers, the proposal fails.

Why not just use deep ensembles or dropout for uncertainty? This is a real threat to the calibration claim. Ensembles give per-voxel spread, but each member makes independent per-voxel guesses, so their samples need not be topologically coherent masks. We will therefore evaluate calibration of structural events, not just voxels: the probability that a segment is connected, checked against how often it actually is. If ensembles are as well calibrated on these events, we will narrow the claim to topology alone.

Why not just train the energy by pseudo-likelihood or MCMC maximum likelihood? Pseudo-likelihood is the single-flip ablation. Our hypothesis is that once continuity potentials are strong, flipping one voxel inside a vessel is always obviously wrong. The learning signal then saturates and says nothing about where the edge or the endpoint belongs. The persistent contrastive divergence ablation tests the alternative of sampling inside the training loop, which the file's evidence suggests is slow and unstable in exponentially large spaces (Bayesian Numerical Integration with Neural Networks; Squared Neural Families). If contrastive divergence matches us at equal compute, the score-matching component is unnecessary and we will say so.

Why not just post-process with classical higher-order solvers? QPBO-type reductions left 92.46% of variables unlabeled with HOCR and 12.46% even with Relax-Expand-Optimize on synthetic higher-order energies (Higher Order Energies for Image Segmentation). They also need hand-set potentials and return no probabilities. We include a hand-designed curvature-plus-length energy optimized by our sampler as a further reference.

Proposed Experiments and Decisive Checks

The main experiment is the degraded-contrast ImageCAS sweep. For every arm we plot Dice, HD95, centerline continuity, number of predicted connected components per vessel tree, and expected calibration error in a 3-voxel band around the boundary, each against contrast severity. The central prediction is that the gap between the full model and both the single-flip and pairwise ablations widens as contrast drops. At the mildest degradation the gap should be near zero, because the image alone suffices there.

Three outcomes would falsify the idea. If single-flip training matches structured edits, the edit-set claim fails. If pairwise cliques match higher-order ones, the higher-order claim fails. If NA-UNETR alone matches us on continuity, the whole approach adds nothing over a good local-attention backbone.

A secondary check replaces synthetic degradation with real non-contrast LAD if LAD-SEG becomes available, or with ETIS-LaribPolypDB in 2D. Since polyps are compact, we expect smaller gains there, and a null result on polyps alongside positive vessel results would itself be informative.

Finally, we report wall-clock training and inference cost relative to the backbone and the gradient variance of each edit family. A model that is only better at ten times the compute should be reported as such.