Knowing When a Boundary Was Seen: Detectability-Gated Skip Connections for Low-Contrast Segmentation

Medical segmentation networks are asked to trace the edges of organs and lesions whose borders are faint: the pancreas against the duodenum, the prostate apex, a flat polyp under glare. In some of these places the image still contains the boundary, just weakly. In others, noise has buried it and no model can recover it. Current networks handle both cases the same way. They draw a crisp, confident contour everywhere, and where the evidence is gone that contour is invented from the network's learned sense of typical shapes. This proposal builds a segmenter that estimates, at each point along the predicted boundary, whether the image actually supports an edge there. It uses that estimate to decide how much fine-detail information to pass through the network's shortcut connections, and it marks each boundary segment as either observed in the image or inferred from shape knowledge.

The opportunity comes from putting two findings side by side. The first is that the shortcut ("skip") connections in encoder-decoder networks copy raw early-layer features, noise included, straight into the output stages. The second is that below a certain contrast-to-noise level, restoration stops being reconstruction and becomes synthesis. Skip connections are exactly where local image evidence enters the decoder, so they are the natural place to apply a recoverability test. The deliverable is both a method and a measurement: a boundary benchmark indexed by physical detectability, on which we can ask whether any segmenter, ours included, knows the difference between seeing an edge and guessing one.

Core Claim

We claim three things. Low-contrast boundary errors in current segmenters split into two kinds that need different treatment. Recoverable-but-weak edges are damaged by noise leaking through unfiltered skips. Unrecoverable edges are hallucinated by modules that force edge predictions. A simple statistical test of local edge detectability, computed from estimated noise level and region intensities on either side of a candidate boundary rather than from image gradients, can tell these two cases apart at test time. Gating skip features by this test and softening the supervision target where the test fails should sharply reduce confident boundary errors in the unrecoverable regime without costing overlap accuracy where edges are visible.

The detectability statistic is a two-sample contrast. Consider a boundary point with $n$ pixels sampled along the local normal on each side. Let $\mu_{in}$ and $\mu_{out}$ be their mean intensities (or mean feature responses), and let $\hat\sigma$ be the estimated noise standard deviation. Then

$$ d = \frac{|\mu_{in}-\mu_{out}|}{\hat\sigma\sqrt{2/n}}. $$

This is the standard z-score for a difference in means, so it approximates how many noise standard deviations separate the two sides. The skip gate at that location is $g=\mathrm{sigmoid}(a(d-\tau))$, with a threshold $\tau$ and slope $a$ learned or set near the conventional detection level. When $g$ is low, the decoder must rely on the bottleneck's global shape context, and the output there is labeled "inferred."

Relation to Prior Work

The skip-filtering line motivates the architecture but does not already contain it. UCTransNet showed that a U-Net without skips can beat one with all skips on GlaS. It replaced the skips with decoder-queried channel cross-attention. CMFCUNet has high-level features query low-level ones pixel by pixel for pancreas CT. SIP-Net multiplies each long skip by a learned sigmoid mask. All three learn their gates purely to maximize overlap with ground-truth masks. None asks whether the incoming features carry recoverable signal, so when contrast falls to zero the gate either suppresses real tissue or passes noise.

Our gate is driven by an explicit physical test rather than by overlap alone. That distinction is also what separates this proposal from the obvious "attention on skips" idea these papers already cover.

The boundary-refinement line supplies the failure we want to measure. Reverse attention in Parallel Res2Net RA, MISNet and CaraNet erases confident foreground so that later layers concentrate on ambiguous edges. We hypothesize that this is precisely the mechanism that produces confident contours where no edge exists. None of these papers reports errors as a function of local detectability, so the hypothesis is untested.

The gradient-based boundary losses constrain our design. Canny-derived contour supervision in the contour-loss U-Net and the edge-alignment term in Boundary-aware InfoMax both anchor to intensity gradients, and gradients fire on noise. This is why our statistic uses region means, which average noise down by $\sqrt{n}$, rather than derivatives, which amplify it.

The restoration literature supplies the conceptual boundary. Cardio-NAFNet argues that denoisers restore rather than create lost morphology. GLEAN and ReFIR show that generative priors fill missing detail with plausible but fabricated content unless they are grounded. We carry that distinction from image restoration into dense segmentation, where none of these works operate.

Research Program

The first component is a detectability-indexed boundary benchmark. We take clean training and test images from PROMISE12 prostate MRI, the NIH pancreas CT set, ACDC cardiac MRI, and Kvasir-SEG/ETIS polyps. We add calibrated degradations at graded levels: Gaussian noise for MRI, Poisson-plus-Gaussian noise for CT, and blur for all modalities. Because the degradation is known, we can compute the true $d$ at every ground-truth boundary point using the clean image and the known noise variance. Boundary points are then binned into detectability bands. The benchmark reports, per band, boundary distance error, boundary-band calibration, and a "confident hallucination rate." That rate is the fraction of boundary-band pixels in low-$d$ bins that are predicted wrong with probability above 0.9.

Clean images already contain naturally low-$d$ segments, such as the prostate apex and pancreas tail. We therefore also compute an estimated $d$ on unmodified images, which lets us report results without synthetic noise.

The second component is the model, the detectability-gated U-Net. It is a standard U-Net or Res2Net-backbone decoder with four changes.

  1. Noise estimation. A robust median-absolute-deviation estimate of noise is taken from high-frequency residuals of the input. This is a classical estimator with no learning.
  2. Local detectability. A coarse first-pass prediction defines boundary normals, and $d$ is computed along them at each skip resolution. At deeper scales we use feature responses projected onto their principal component in place of raw intensities.
  3. Detectability gating. The gate $g$ multiplies the skip features in a band around the coarse boundary. Away from the boundary, a conventional learned gate of the SIP-Net type is kept.
  4. Softened supervision. In ground-truth boundary regions whose true $d$ falls below $\tau$, the hard mask label is replaced with a label blurred across the normal. The blur width is proportional to $\hat\sigma/|\mu_{in}-\mu_{out}|$ times the edge blur width, which is how the lower bound on edge-position error for a noisy blurred step scales. This teaches the network to widen its probability band exactly where localization is physically impossible, instead of rewarding a crisp guess.

The output is a mask, a per-pixel probability, and an observed/inferred flag along the contour.

The third component is analysis. We will test whether reverse-attention modules measurably increase the confident hallucination rate in low-$d$ bins relative to a plain U-Net, which is the untested prediction behind Gap 2. We will also check whether estimated $d$ tracks true $d$ well enough to be usable, reporting their correlation per modality.

Baselines

The naive reference is a plain U-Net with all skips. Next to it we run a skip-free U-Net, to test whether the skip paradox reported by UCTransNet on GlaS reappears in low-$d$ bins specifically. UCTransNet itself reports 90.18% Dice on GlaS against 85.45% for vanilla U-Net.

The strongest learned-gating competitors are SIP-Net, which reports 91.42% DSC on PROMISE12 and 83.9 ± 4.5% DSC on TCIA Pancreas CT-82, and CMFCUNet, which reports 86.30% DSC on NIH pancreas. They are the strongest competitors on this question because they already filter skips with learned spatial gates. If a learned gate implicitly discovers recoverability, it is one of these.

For polyps, the strongest boundary-refinement competitor is CaraNet, which reports 74.7% mDice on ETIS. MISNet reports 0.903 mDice on Kvasir-SEG. These represent reverse attention at its best.

For uncertainty, we run a five-member deep ensemble of the plain U-Net with temperature scaling, which serves as the standard way to flag unreliable pixels. There is no reported result on Lacuna for deep ensembles on these benchmarks, so we will establish it ourselves.

For fairness, every method is retrained by us on identical splits and identical degradation-augmented training data, with matched backbones where the architecture allows. Each gets the same hyperparameter search budget, roughly twenty trials, tuned on the same validation fold. We also report each method's published clean-data numbers to confirm that our reimplementations are faithful.

Two ablations isolate our ingredients. The first replaces the $d$-driven gate with a learned sigmoid gate of identical capacity, trained on the same augmented data. The second keeps the gate but drops the softened low-$d$ supervision.

Beating the baselines means three results together. Our model must cut the confident hallucination rate in the lowest two detectability bands by a clear margin, at least a halving relative to the best competitor. It must match that competitor's Dice and 95% Hausdorff distance within noise in high-$d$ bands and overall. And it must produce observed/inferred flags whose "inferred" label predicts boundary errors better than ensemble variance does.

Why not just…?

Why not just use a deep ensemble or Monte Carlo dropout to flag uncertain boundaries? This is the real threat. If ensemble disagreement already rises in unrecoverable regions, the flagging half of our contribution is unnecessary. Our reason to doubt it is that ensemble members trained on the same data share the same shape prior. They can agree on the same invented contour, which would give low variance in exactly the cases that matter. The benchmark settles this directly: we measure how well ensemble variance and our flag each predict boundary error within low-$d$ bins. If the ensemble matches our flag there, we will report that result and narrow the claim to the gating and softened-supervision gains on accuracy and calibration.

Why not just train a learned skip gate such as SIP-Net's with heavy noise augmentation? A learned gate might absorb the recoverability test on its own. The first ablation above is built for this: same capacity, same augmentation, gate driven by overlap loss instead of by $d$. The argument in our favor is that an overlap loss rewards committing to the most likely contour even where the edge is invisible, so nothing pushes a learned gate toward abstention. If the learned gate matches ours on the low-$d$ hallucination rate, the explicit statistic has no value and we will say so.

Why not just remove skip connections, or denoise the image first? Removing skips trades hallucination for blur everywhere. The skip-free baseline will show whether it loses high-$d$ boundary accuracy, as the spleen Hausdorff results from the contour-loss U-Net suggest it will. Generative pre-denoising moves the invention into the input image, as GLEAN's comparison with PULSE and ReFIR's analysis of hallucination in large restoration models both show. We include a classical, non-generative denoiser before a U-Net as an extra baseline. We expect it to help mid-$d$ bins but not the lowest, where no signal remains to denoise.

Proposed Experiments and Decisive Checks

The first decisive check is diagnostic and precedes any method claim. On the degraded benchmark, do existing methods show a sharp rise in confident hallucination rate as true $d$ falls below roughly 2–3? And is that rise larger for reverse-attention models than for a plain U-Net? If errors in low-$d$ bins turn out to be uncertain rather than confident, the premise fails and the method is not needed.

The second check is whether the estimated $d$, computed without ground truth, correlates with the true $d$ strongly enough (rank correlation above about 0.7) to drive a gate. If it does not, we fall back to using $d$ only as a training-time label for the softened supervision.

The third is the head-to-head comparison against CaraNet, SIP-Net, CMFCUNet, the ensemble and the ablations. Performance is reported per detectability band on the degraded sets, and on the unmodified PROMISE12, NIH pancreas, ACDC, Kvasir-SEG and ETIS sets, with Dice, 95% Hausdorff distance, boundary-band calibration error and confident hallucination rate.

The fourth is a small reader study on ETIS and pancreas cases. Clinicians judge whether contours flagged "inferred" coincide with regions they themselves could not delineate. This checks that the flag corresponds to something humans also cannot see, rather than to an artifact of the synthetic noise model.