Renormalized Convolution on the Uncertainty Band for Small, Faint Targets

When a segmentation network must find something tiny and faint, such as a flat polyp, a hair-thin communicating artery, or a low-contrast defect, the few pixels that matter get averaged away. Every convolution window near the target is dominated by background, so the target's signal is diluted before the loss ever sees it. Boundary-weighted losses change where the gradient points but leave this averaging in the forward pass unchanged. We propose a two-pass network. The first pass makes a coarse prediction. The second pass refines it with partial convolutions, which are convolutions that ignore pixels marked invalid by a companion mask and rescale the result by how many valid pixels were in the window. The twist is that the mask is not a map of missing data. It is the first pass's own uncertainty band: confidently-background and confidently-foreground pixels are declared "unobserved," and features near the ambiguous margin are computed only from the ambiguous margin, at full magnitude.

The closest existing mechanism is reverse attention. It already erases confident regions by multiplying features with a soft mask, as in PraNet-style refiners (Parallel Res2Net-based Network with ReverseAttention for Polyp Segmentation, CaraNet). Our claim is narrower and testable. Erasure without renormalization shrinks feature magnitude in proportion to how much of each window the band occupies. For small, rare targets that fraction is tiny, so reverse attention fails exactly where it is needed. Renormalization removes that dependence. The benefit of the proposal should therefore grow as target size and contrast shrink, and vanish for large, high-contrast objects.

Core Claim

Write the first-pass foreground probability as $p$. Define a soft band weight $u = \max(\epsilon, 4p(1-p))$, which is near 1 where the model is unsure and near a floor $\epsilon$ where it is confident. The refinement layers compute

$$y = W^\top (X \odot U)\,\frac{|\Omega|}{\sum_{\Omega} U + \delta} + b,$$

In this expression, $X$ is the input feature patch over window $\Omega$ and $U$ the corresponding band weights. $|\Omega|$ is the number of positions in the window, $W, b$ are the kernel and bias, and $\delta$ is a small stabilizer. The mask propagates by max-pooling $U$ between layers, following the progressive update of Image Inpainting for Irregular Holes Using Partial Convolutions.

Reverse attention computes the same product without the ratio. Its output therefore scales roughly with the local band fraction $\rho = \sum_\Omega U / |\Omega|$. For a target a few pixels wide, $\rho$ can be a few percent, and the refined feature is correspondingly faint. Our prediction follows directly from this factor: the gap between the two should widen as $\rho$ falls.

The soft weights are a deliberate response to the tension the file identifies between hard binary masks and continuous distance fields. Sparsity Invariant CNNs argues that strict masking is what stops feature pollution. Shape-Aware Organ Segmentation by Predicting Signed Distance Maps shows that discretized targets produce jagged, high-Hausdorff contours. A soft mask with an explicit normalizer keeps the invariance and avoids the discretization.

Relation to Prior Work

Several papers supply the method. Uhrig et al. (Sparsity Invariant CNNs) introduced mask renormalization for sparse depth. They showed that simply concatenating the mask as a channel is worse: 1.94 m versus 1.81 m RMSE on KITTI. Liu et al. extended the idea to inpainting holes. Both treat the mask as given by the sensor or the user. We derive it from the model's own uncertainty, which is the unexplored link the file calls Gap 1.

Gated and attention variants replace hard masks with learned soft ones: MagaConv, gated convolution for panoramas, and learnable bidirectional attention maps. They learn the gate end to end. We tie it to calibrated first-pass uncertainty so that its meaning is fixed and it can be ablated.

Segmentation-Aware Convolutional Networks masks each neighborhood by distances in a learned embedding. That construction depends on embeddings separating foreground from background, which is exactly what collapses under low contrast. PA-SAM (PConv-Based Fine-Tuning for Defect Segmentation) uses partial convolutions only as an efficient adapter over channels, with no spatial uncertainty mask and no imbalance setting.

Other work motivates the proposal. Loss-level remedies for thin, rare structures improve results, but they leave the forward pass dense:

NA-UNETR also constrains the design. Its finding that local attention beats global attention on faint arteries suggests that restricting aggregation helps. CaraNet argues the opposite, that small objects need wide context to avoid false positives. Our design keeps the first pass fully contextual and restricts only the refiner, so both positions are respected.

Recursive refiners feed predictions back through dense convolutions by summation or concatenation: RecycleNet and the left-atrium pipeline of Combating Uncertainty with Novel Losses. Uhrig et al. found this kind of input "sparsity-blind." We test whether that finding transfers from missing data to uncertainty.

Research Program

We will build the refiner as a drop-in second stage on a fixed backbone. We use nnU-Net for 3D vascular data and a Res2Net-50 encoder for polyps, matching MISNet and CaraNet. The first-pass probabilities are temperature-calibrated on a validation split so that $u$ means the same thing across datasets.

The refiner is trained with the same composite loss used by the strongest baseline for that dataset. This choice isolates the architecture from the loss. We add a variant that restricts a boundary loss to the same band, in the spirit of the trimap supervision in PostureHMR, to test whether coupling mask and loss adds anything.

The floor $\epsilon$ matters for recovering missed targets. With $\epsilon = 0$, a target the first pass calls confident background can never be recovered. We treat $\epsilon$ as a studied hyperparameter rather than a tuned nuisance.

The analysis has three parts. First, we measure feature magnitude and signal-to-noise inside the band as a function of $\rho$, for our refiner and for reverse attention. This directly tests the proposed mechanism. Second, we stratify every metric by target area fraction and by local contrast-to-noise. Third, we run a controlled synthetic study. Lesions are planted into real target-free background patches from the polyp and defect datasets, with contrast and area swept on a grid, so the predicted interaction can be observed without dataset confounds.

Baselines

The naive reference is the backbone alone, trained with Dice plus cross-entropy and no refinement stage. Every other method must beat it at matched parameter count. We also run the backbone plus a refiner of identical depth built from ordinary convolutions, so that "more layers" is not mistaken for "masked layers."

On polyps, the strongest existing methods are CaraNet, which reports 74.7% mDice on ETIS-LaribPolypDB (CaraNet), and MISNet, which reports mDice of 0.903 on Kvasir-SEG and 0.918 on CVC-ClinicDB. Both combine a strong context pyramid with boundary or reverse attention, and ETIS is exactly the small, concealed regime we target. We retrain both from their published configurations on our splits and give them the same tuning budget as our method. Beating CaraNet means a statistically significant mDice gain on ETIS over multiple seeds, with the gain concentrated in the smallest-area stratum.

On vessels, the strongest competitor is cbDice with nnU-Net. It reports 43.4% Dice on TopCoW small communicating arteries, against 38.5% for clDice (Centerline Boundary Dice Loss for Vascular Segmentation). We keep cbDice as the loss and add our refiner, so any gain is attributable to the forward pass. On LAD-SEG, NA-UNETR reports 45.64% Dice against nnU-Net's 42.54% (NA-UNETR). We use it as the attention-based competitor under the same 20-scan protocol and the same ImageCAS pretraining.

The key ablation is reverse-attention refinement with the identical band $U$ and identical layers, differing only in the renormalizing ratio. A second ablation concatenates $U$ as an input channel to an ordinary convolution refiner, which reproduces Uhrig et al.'s negative control in our setting. A third replaces $U$ with a fixed morphological band around the first-pass contour, as in the erosion bands of Bokhovkin & Burnaev. This tests whether uncertainty is better than geometry as the mask source.

Why not just…?

Why not just use reverse attention, which already erases confident regions? This is the most serious threat, because the two methods differ by one scalar per location. The ablation above settles it. If renormalization matters, the gap over reverse attention should be near zero on large Kvasir polyps and grow monotonically as area fraction falls. The synthetic grid should show the same pattern along the size axis. If the gap is flat across strata, the renormalization story is wrong, and we will report the method as an equivalent reparameterization of reverse attention.

Why not just concatenate the uncertainty map as an extra channel? Uhrig et al. showed that in sparse depth a concatenated mask leaves convolution sparsity-blind: 1.94 m versus 1.81 m RMSE (Sparsity Invariant CNNs). There is no evidence either way for uncertainty masks, because a learned network could plausibly learn to gate on a channel it sees at every layer. The concatenation ablation is therefore run at every refiner layer, not just the input, to give it its best chance.

Why not just use a stronger boundary or imbalance loss with ordinary convolutions? cbDice's 4.9-point gain on small arteries shows that loss design is powerful. The left-atrium study shows focal and overlap losses lifting Dice from 82.01% to 91.54% (Combating Uncertainty with Novel Losses). We never pit our architecture against a weaker loss: every comparison holds the best available loss fixed. If our refiner adds nothing on top of cbDice on TopCoW, the claim fails for tubular structures.

Why not just crop a region of interest and run a second network, as in two-stage rare-event pipelines? Cropping rebalances pixel counts globally. It is the approach of the left-atrium pipeline's detector stage and of DC-Prophet for telemetry. But inside the crop, a thin margin is still diluted by the surrounding tissue. We include an oracle-crop baseline built from ground-truth bounding boxes. Beating even that oracle would show that the effect is local dilution rather than global imbalance.

Proposed Experiments

The decisive experiment is the synthetic contrast-by-size grid. Planted lesions span area fractions from roughly 10% down to under 0.1% of a patch, crossed with several contrast-to-noise levels. The prediction is an interaction: our refiner's advantage over reverse attention and over the dense refiner is largest in the small-and-faint corner and near zero in the large-and-bright corner. We will also verify that the measured band feature magnitude under reverse attention tracks $\rho$ while ours does not. If the mechanism is present but the accuracy gap is not, the proposal is wrong about why small targets fail.

On real data we evaluate on Kvasir-SEG, CVC-ClinicDB, ETIS-LaribPolypDB, SD-Saliency-900 and Tire-Seg, TopCoW 2023, and LAD-SEG. The metrics are mDice, normalized surface distance, HD95, and boundary F1. We add lesion-level detection recall at fixed false positives per image, because overlap metrics distort badly on few-pixel targets, and Betti-number errors on vessels. Every metric is stratified by target area.

To probe rarity directly, we tile polyp and defect images into patches and subsample target-containing patches to prevalences of 1:10, 1:100 and 1:1000. We then report free-response operating curves (detection sensitivity plotted against false positives per image). This checks whether the band-restricted refiner avoids the false-positive explosion that cost-weighting causes. Our expectation is that it does, because confidently-background regions are suppressed by the floor $\epsilon$. Finally, we measure how often targets missed entirely by the first pass are recovered as a function of $\epsilon$. That number is the honest cost of letting the model decide what it ignores.