Correction-Weighted Boundary Losses for Iterative Segmentation of Low-Contrast Structures

Some segmentation networks draw a mask in several passes. Each pass receives the previous draft and improves it, much like a radiologist tracing an outline and then going back over it. A common way to get crisper outlines is a boundary-weighted loss, a training objective that penalizes mistakes near an object's edge more heavily than mistakes in its interior. This note argues that combining the two can backfire on low-contrast images, such as camouflaged polyps in colonoscopy or ill-defined stroke lesions on MRI. When every pass is rewarded for producing a sharp edge, an early pass learns to commit to a confident outline in places where the image gives little evidence. Later passes then treat that outline as fact and sharpen it further. Image-restoration pipelines show the same failure: a super-resolution stage magnifies the ringing left by an upstream demosaicing stage because it cannot tell artifact from detail (Deep Residual Network for Joint Demosaicing and Super-Resolution). We expect iterative segmenters to do the same with boundary errors.

We propose to measure this "boundary lock-in" directly and to fix it by changing what the boundary loss rewards at each pass. The standard approach puts extra weight on every pixel near the true edge at every pass. Our loss instead puts extra weight where the incoming draft's boundary is wrong. The refinement passes are therefore trained to reverse inherited edge errors rather than to sharpen whatever they are handed. Inside a thin band where annotations are themselves uncertain, targets are softened. The claim is testable. If it holds, gains should appear specifically in low-contrast boundary regions and on unseen colonoscopy domains, and should not appear when the same extra weight is spread uniformly over the boundary band.

Core Claim

Iterative refinement and boundary-weighted losses have each been shown to help on ambiguous medical boundaries. Nobody has examined how they interact. RecycleNet feeds latent features and predictions back into earlier layers so the network can "ponder" and refine its decision. It reports that refinement extends beyond the number of passes seen in training, but it uses no boundary-aware objective. Boundary objectives are designed and evaluated in single-pass networks. These include the distance-map integral of Boundary loss for highly unbalanced segmentation, signed-distance regression in Shape-Aware Organ Segmentation by Predicting Signed Distance Maps, the active-contour-style Active Boundary Loss, and the fixed 10× band weight of Chakravarty et al..

Our hypothesis has two parts. First, applying a standard boundary loss at every pass increases the rate at which early boundary errors in low-contrast regions survive to the final pass, relative to region-only supervision. Second, a loss that weights the boundary band by disagreement between the incoming draft and the label lowers that survival rate and improves final boundary accuracy without hurting interiors.

The ingredient that is new here is the correction weighting. Yeung et al.'s soft boundary band (Incorporating Boundary Uncertainty into loss functions) is reused as a method component. It shows that hard boundary penalties overfit to subjective annotations on fuzzy edges and that a narrow soft-target band fixes this at almost no cost. It is a single-pass method with no notion of an incoming draft.

Mechanism

Let $p_t$ be the predicted foreground probability map after pass $t$ of $T$. Let $y$ be the label, and let $B$ be a morphological band a few pixels wide around the labeled boundary. During training, the draft $p_{t-1}$ fed into pass $t$ is the model's own detached output, never the label. The refinement passes therefore see the kind of errors the model actually makes. The loss at pass $t$ is

$$ L_t = L_{\text{region}}(p_t, y) + \lambda \sum_{i \in B} w_{t,i}\, \ell(p_{t,i}, \tilde y_i), \qquad w_{t,i} = 1 + \gamma\, \mathbb{1}\left[\, \mathbb{1}[p_{t-1,i} > 0.5] \neq y_i \,\right], $$

The symbols are defined as follows. $L_{\text{region}}$ is the Dice plus cross-entropy loss used by nnU-Net. $\ell$ is per-pixel cross-entropy. $\tilde y_i$ are Yeung-style soft targets inside $B$. $\lambda$ sets overall boundary emphasis. $\gamma$ sets how much more an inherited mistake costs than a pixel the incoming draft already had right. For the first pass, $p_0$ is an empty draft and $w_{1,i}=1$. Following the curriculum lesson from Kervadec et al. and ABL, which found that boundary terms cannot localize objects from scratch, $\lambda$ is ramped in after the region loss has stabilized. The total loss sums $L_t$ over passes.

Research Program

The work proceeds in three stages. The first is a diagnostic study that establishes whether lock-in exists. We take nnU-Net (nnU-Net) as the base network and add RecycleNet-style recycling with $T$ passes. We train it under region-only loss, standard boundary-band loss at every pass, and boundary loss on the final pass only. At inference we track every boundary pixel across passes.

Two quantities summarize the dynamics. The persistence rate is the fraction of pixels misclassified within the band at pass 1 that are still misclassified at pass $T$. The amplification curve is Hausdorff distance as a function of pass index, extended past $T$ to test RecycleNet's claim of stable convergence. Each labeled boundary pixel is assigned a contrast stratum. The stratum is computed from the intensity difference across the boundary normal in the raw image. All results are reported per stratum, because the hypothesis predicts harm concentrated in the lowest-contrast tertile.

The second stage trains the correction-weighted loss and asks whether it changes these dynamics. The third stage transfers the loss to a feed-forward cascade. PraNet builds a coarse global map and then applies reverse-attention stages that erase confident foreground so later stages focus on the remaining edge. It is a natural test, because its stages also inherit upstream drafts. Its drop from Kvasir to ETIS is consistent with early truncations of ambiguous polyp edges being locked in downstream. We supervise each reverse-attention stage's side output with the correction-weighted loss, computed against the previous stage's map.

Baselines

The naive reference point is nnU-Net with its default Dice plus cross-entropy loss and a single pass. It establishes whether iteration and boundary weighting are worth anything at all. It is run with the dataset-specific configuration nnU-Net derives automatically. nnU-Net's results argue that this reference is hard to beat: it won both phases of the Medical Segmentation Decathlon and reached 88.1% average Dice on the BCV abdominal challenge.

The strongest existing methods differ by benchmark, and each is reproduced on our data splits with the same augmentation and epoch budget as our model. On polyps, the reference is PraNet trained on the standard Kvasir/ClinicDB split. PraNet reports a mean Dice of 0.898 on Kvasir and 0.628 on unseen ETIS (PraNet). MISNet (Kang et al.) reports mDice 0.903 on Kvasir and 0.918 on CVC-ClinicDB, and a 16.8% drop on ETIS. No absolute ETIS Dice for MISNet was found on Lacuna, so we will rerun it rather than quote it. On stroke and white-matter lesions, the reference is Generalized Dice plus boundary loss, which reports Dice 0.656 on ISLES and 0.748 on WMH (Kervadec et al.). On DRIVE, the reference is Yeung's boundary-uncertainty Dice at DSC 0.8141. For RecycleNet itself, no reported result was found on Lacuna for these benchmarks. We implement recycling on the same nnU-Net backbone, so the recycled and non-recycled models share architecture and compute up to the extra passes.

Three ablations isolate the key ingredient. The first gives a uniform band weight equal to the average total weight our correction rule assigns. It tests whether gains come from where the weight goes or merely from how much there is. The second applies the boundary loss on the final pass only. The third applies Yeung's soft band at every pass without correction weighting.

Every variant receives the same hyperparameter search budget over $\lambda$, band width, and ramp schedule, and is trained with three seeds. "Beating" a baseline has a precise meaning. Our final-pass Hausdorff distance and boundary F-score in the low-contrast stratum must be better than every ablation beyond seed variance. Persistence rate must be lower. Region Dice must not fall. On ETIS we must exceed our rerun of PraNet's 0.628 using PraNet's own training data.

Why not just…?

Why not just apply the boundary loss only at the final pass? This is the most serious threat. If lock-in comes from intermediate sharpness pressure, removing that pressure might be enough. The diagnostic study tests it directly. If final-pass-only matches correction weighting on persistence and low-contrast boundary error, we will report that the simpler fix suffices, and the contribution reduces to the diagnosis. Our reason to expect a gap is that final-pass-only supervision never tells the intermediate passes which inherited errors to undo. They receive only diffuse region gradients through the recycling path.

Why not just add random noise to the fed-back draft during training? Generic perturbation teaches robustness to noise, not correction of the structured errors this model makes. Those errors are shifted or truncated contours in low-contrast regions. We include a variant that perturbs the draft with random contour erosion and dilation of matched magnitude. If it closes the gap, the lesson is that error-aware training data matters more than the loss. That would still be informative, but it would weaken our claim.

Why not just use nnU-Net and post-process the boundaries? Post-processing can smooth or shift a contour, but it cannot restore structure an upstream stage erased. Even for refinement, ABL reports that end-to-end boundary training beat the post-hoc refiner SegFix on Cityscapes boundary F-score. We still run CRF smoothing on the nnU-Net baseline so the comparison is explicit.

Why not just smooth labels everywhere? Yeung et al. found that global soft labeling injects uncertainty into unambiguous interiors and underperformed localized boundary uncertainty on DRIVE, 2018 Data Science Bowl, and CVC-ClinicDB. We therefore keep softening local to the band and do not rerun this alternative beyond one sanity check.

Proposed Experiments

The polyp experiments follow PraNet's protocol. We train on Kvasir and CVC-ClinicDB and test on held-out Kvasir and ClinicDB as seen domains, and on ETIS, CVC-ColonDB, and CVC-300 as unseen domains. Lesion experiments use ISLES and WMH. DRIVE probes thin structures, where boundary pixels form most of the object. We report Dice, 95th-percentile Hausdorff distance, and boundary F-score, all stratified by local contrast. We also report persistence rate and amplification curves out to twice the trained number of passes.

Three outcomes decide the question. First, standard every-pass boundary loss must show higher persistence than region-only training in the low-contrast stratum. If it does not, the motivating failure does not exist and we report a negative result. Second, correction weighting must beat the uniform-weight ablation at matched total weight. This is the test that the gain comes from aiming the boundary loss at inherited errors and not from more boundary emphasis. Third, gains must be larger on unseen polyp domains than on seen ones. That pattern would show the method addresses upstream truncation under domain shift rather than fitting the training distribution more tightly.