Segmentation networks tend to draw masks that are too small around objects that are both tiny and faint, such as flat colon polyps or small tumor cores. We propose that this is not a generic "hard example" problem but a specific, measurable bias with a simple cause. A single output score is asked to do two jobs at once. It must decide whether a lesion is present at all, and it is pushed toward "background" everywhere because background dominates the pixels. It must also mark where the lesion's edge lies. That constant push toward background moves every predicted edge inward. The shift is largest exactly where the image gives the network the least evidence: at low-contrast boundaries, and on small objects whose signal was blurred by the network's internal downsampling. Tiny, faint lesions shrink the most, and past a point they disappear.
This is the same pathology that sparse autoencoders suffer under an L1 penalty. There, a sparsity penalty that is needed to decide which features fire also shrinks how strongly they fire. Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders removed that shrinkage by splitting detection from magnitude estimation. We carry this split into segmentation. An object-level gate absorbs the background prior and controls false positives. A separate, unpenalized "extent" field locates the boundary. Because the gate acts per object rather than per pixel, the extent field stays smooth, and boundary losses still pass gradients through it. The contribution has two parts: a diagnostic, a signed shrinkage ratio for segmentation stratified by object size and contrast, and an architecture change. Both are tested on polyp and brain-tumor benchmarks where small, low-contrast targets are known to break current models.
Core Claim
Write the network's pre-sigmoid output as $z(x) = s(x) - b$. Here $s(x)$ is the image evidence for foreground at pixel $x$, and $b > 0$ is the offset that class imbalance and the loss induce. The predicted boundary is the level set $z = 0$, while an unbiased boundary sits where the evidence crosses its own midpoint. Expanding $s$ to first order across the edge gives the inward displacement
$$\delta \approx \frac{b}{|\nabla s|},$$
where $|\nabla s|$ is the steepness of the evidence across the boundary. Three consequences follow, one from each direction in this note. Low physical contrast makes $|\nabla s|$ small, so the same offset moves the edge further. Downsampling blurs a small object's evidence, which lowers both its slope and its peak. Once the peak of $s$ falls below $b$, the object vanishes entirely rather than merely shrinking. The claim is testable. Predicted-to-true area ratio should fall below one as contrast drops and as object size drops, with an interaction between the two. A design that removes $b$ from the extent field while keeping it in an object-level gate should push that ratio back toward one without raising false positives.
Relation to Prior Work
The shrinkage literature supplies the diagnosis and the fix, but none of it works in images. Confidence Intervals and Hypothesis Testing for High-Dimensional Regression shows that L1-regularized estimates are pulled toward zero and corrects them with a residual-based term. We borrow its idea of a post-hoc offset correction as one baseline, not its asymptotic theory, whose sparsity conditions have no obvious analogue for dense pixel maps. The Gated SAE paper supplies the method we reuse: a separate detection path carrying the penalty and an unpenalized magnitude path. It also supplies a scalar bias metric, $\gamma$, the ratio of reconstructed to true magnitude, which we adapt to area. It works only on one-dimensional language-model activations and says nothing about spatial structure or boundaries.
Volumetric Bias in Segmentation and Reconstruction: Secrets and Solutions both motivates and constrains the proposal. It proves that classical likelihood-based segmentation energies hide a bias on segment sizes, and it cuts GrabCut error from 14.41% to 7.87% by removing that bias. It also warns of a "descriptive model trap": removing the bias from very expressive models lets one segment swallow the image. We take the warning seriously. That is why we do not remove the background prior; we move it into the object-level gate, where it still blocks degenerate all-foreground solutions.
The small-object and boundary architectures are the competitors, not the source of the method. CaraNet: Context Axial Reverse Attention Network for Segmentation of Small Medical Objects combines dilated multi-scale features with reverse attention, which erases confident interiors to focus later layers on edges. Parallel Res2Net-based Network with ReverseAttention for Polyp Segmentation uses reverse attention followed by a size-threshold "polarization" step. Multi-scale Information Sharing and Selection Network with Boundary Attention for Polyp Segmentation (MISNet) adds an explicit boundary stream. Gated-SCNN: Gated Shape CNNs for Semantic Segmentation adds a full-resolution shape stream and a loss aligning mask gradients with edges. All four add capacity or supervision at the boundary, yet all still read the edge off a single score that carries the background offset. Our prediction is that they reduce $\delta$ by raising $|\nabla s|$ but leave $b$ in place. Reverse attention may also make tiny lesions worse, since a lesion that is almost all boundary has little interior to protect from erasure.
LIP: Local Importance-Based Pooling attacks the resolution side with learned importance weights in pooling, raising COCO small-object AP for Faster R-CNN from 21.9% to 24.0%. It addresses the peak-attenuation half of the mechanism, not the offset half, so we treat it as a complementary factor in a factorial experiment rather than a rival.
One tension needs addressing directly. The research file notes that step-like gates break the smooth gradients that boundary losses such as Gated-SCNN's need. Our design avoids this by gating at the object level with a coarse presence map. The pixel-level extent field is never passed through a step function, so boundary losses apply to it unchanged.
Research Program
The first deliverable is a measurement. For every ground-truth connected component, we compute the signed area ratio $\gamma_{\text{seg}}$, the sum of predicted soft mask values over the component's neighborhood divided by its true area. We also compute the mean signed normal displacement of the predicted contour. Components are stratified by true area and by a contrast score: the mean intensity or colour difference between thin bands just inside and just outside the annotated edge, normalised by local texture variance. If $\gamma_{\text{seg}} < 1$ grows worse jointly with smaller size and lower contrast in standard models, the phenomenon is real and localized. If $\gamma_{\text{seg}}$ is near one everywhere and errors are symmetric, the premise fails and the project stops here.
To test the mechanism rather than just the symptom, we build a controlled synthetic set. Real polyp masks are blended into lesion-free mucosa at chosen scales and chosen edge contrasts, using alpha-blending with a tunable softness. We then check whether measured inward displacement scales with the inverse of the network's own logit slope across the edge, as the displacement equation predicts. We also check whether the vanishing point, the size below which recall collapses, moves with contrast as predicted.
The method is a two-head decoder that can sit on any existing segmentation backbone. The presence head predicts a coarse map of lesion centres or instances and is trained with the ordinary imbalanced loss, so it carries the background prior. The extent head predicts a full-resolution field whose zero level set is the boundary. It is trained with a class-balanced loss restricted to a band around each true or detected object, so background dominance never enters its offset. Any boundary loss, including Gated-SCNN's gradient-alignment term, can be added to this head. The final mask is the extent field's positive region inside objects the presence head accepts.
Following the Gated SAE, we will test a weight-tied variant in which both heads share most parameters and differ only by a per-channel rescaling. This checks that gains come from separating the roles, not from extra capacity.
Baselines
The naive reference point is a plain encoder–decoder trained with the combined binary cross-entropy and Dice loss used throughout the polyp literature, thresholded at 0.5. It is the configuration the displacement equation says should shrink small faint lesions, and it anchors $\gamma_{\text{seg}}$. The strongest existing methods are CaraNet and MISNet. CaraNet reports mDice 74.7% on ETIS and 63.1% on the BraTS 2018 necrotic and non-enhancing tumor core subset, the small-target regime this proposal targets. MISNet reports mDice 0.903 on Kvasir-SEG and 0.918 on CVC-ClinicDB, and a degradation on ETIS of 16.8% against over 38% for prior models. For CaraNet on Kvasir-SEG and CVC-ClinicDB individually, no reported result was found on Lacuna.
We will retrain both with their public configurations on the same training images, with the same input resolution and schedule. Our two-head decoder will be attached to the same backbone, so parameter counts differ only by the extent head. Every method gets the same hyperparameter search budget on a held-out validation split.
Beating them means two things. First, higher mDice on ETIS and on the BraTS small-core subset. Second, and more important for the claim, $\gamma_{\text{seg}}$ closer to one and higher component-level recall in the smallest, lowest-contrast bin, at matched false positives per image. A gain in average mDice alone, driven by large polyps, would not count.
The decisive ablations isolate the separation of roles. The first applies the balanced extent loss to a single head with no gate; the prediction is that false positives rise. The second adds the gate but trains the extent head with the imbalanced loss; the prediction is that shrinkage persists. The third is the weight-tied variant. Finally, a post-hoc offset correction in the spirit of the de-biased Lasso tests whether training-time separation is needed at all. It estimates $b$ on validation data and adds it back to the logits before thresholding.
Why not just…?
Why not just lower the threshold, or tune it per dataset? This is the most serious alternative, because a global downward shift in threshold is exactly a correction of a constant $b$. Its weakness is that the same shift applies to background pixels, so recovering faint edges should cost false positives across the whole image. The displacement equation also says the needed correction depends on local slope, which a single threshold cannot adapt to. We will compare against the best global threshold chosen on validation data and against an oracle per-object threshold chosen with ground truth.
If the tuned global threshold matches our method on the joint tiny-object recall and false-positive curve, the architectural claim is wrong and the finding reduces to "segmentation models are miscalibrated at 0.5." If only the per-object oracle matches, that confirms the gate's role: our presence head is a learned, non-oracle version of that per-object decision.
Why not just use max pooling, dilated convolutions, or LIP to keep small objects from being blurred? These raise the peak and slope of $s$, which by the displacement equation reduces $\delta$ but does not remove the offset $b$. CaraNet's dilated pyramid and LIP's small-object gain show this route helps. We will run a two-by-two design crossing the downsampling operator (strided versus LIP) with the decoder (single head versus two-head). If resolution fixes alone bring $\gamma_{\text{seg}}$ to one in the smallest bin, our component is unnecessary. We expect the two effects to add.
Why not just add a boundary loss or reverse attention, as MISNet and CaraNet do? Boundary supervision sharpens the edge, and so again acts on slope rather than offset. Reverse attention also risks erasing lesions that have no interior. The file's own reading of CaraNet is that it fails when the coarse decoder misses the lesion. We will report $\gamma_{\text{seg}}$ and tiny-object recall for these models directly. We will also add Gated-SCNN's gradient-alignment loss to our extent head, which our design permits because the extent head is never gated at pixel level. That tests whether the two ideas stack rather than compete.
Why not just remove the size prior entirely, as the volumetric-bias paper does? That paper itself shows the danger for expressive models, which is why the balanced-loss-without-gate ablation exists. We expect it to increase false positives or whole-image leakage, showing the prior must be kept and relocated rather than deleted.
Proposed Experiments
Training and testing follow the polyp protocol used by CaraNet and MISNet: train on Kvasir-SEG and CVC-ClinicDB, and evaluate on those plus CVC-ColonDB, CVC-300 and ETIS, with ETIS as the primary small, low-contrast test. BraTS 2018 provides a second modality, using the necrotic and non-enhancing core, where CaraNet's 63.1% is the reference. The synthetic blended-polyp set tests the mechanism under controlled size and contrast.
Metrics are mDice and a boundary F-score for comparability. The primary endpoints are $\gamma_{\text{seg}}$, signed contour displacement, and component-level recall at fixed false positives per image, each reported per size-by-contrast bin with bootstrap confidence intervals over images.
Two checks decide the project. The first is whether standard models show $\gamma_{\text{seg}}$ well below one specifically in the small, low-contrast bin, with displacement tracking inverse logit slope on the synthetic set. Without that, there is no bias to remove. The second is whether the two-head decoder beats the best tuned global threshold on tiny-object recall at matched false positives. If it does not, the honest conclusion is a calibration result, not an architecture, and we would publish the diagnostic alone.
References
- Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders
- Confidence intervals and hypothesis testing for high-dimensional regression
- Volumetric Bias in Segmentation and Reconstruction: Secrets and Solutions
- CaraNet: context axial reverse attention network for segmentation of small medical objects
- Parallel Res2Net-based Network with ReverseAttention for Polyp Segmentation
- Multi-scale information sharing and selection network with boundary attention for polyp segmentation
- Gated-SCNN: Gated Shape CNNs for Semantic Segmentation
- LIP: Local Importance-Based Pooling