Texture Evidence for Where the Scribbles Stop

When experts label medical images with a few quick strokes ("scribbles") instead of tracing full outlines, they tell a segmentation network what is inside each structure but nothing about where one structure ends. In colonoscopy, pancreas CT, or scarred heart muscle, the missing boundary information is hardest to recover, because neighbouring tissues often have nearly the same brightness. Current scribble methods fill the gap by spreading labels outward along intensity similarity, and so they leak across exactly the boundaries that matter. Yet tissues that match in brightness frequently differ in texture: one region may be grainy and the other smooth, or one may be lobulated and the other uniform. Classical texture analysis showed decades ago that such boundaries can be found from differences in local statistical spread even when mean intensity is identical.

This note proposes computing a boundary evidence map from the scribbles themselves. For each pixel, the map records how strongly the local texture statistics on one side differ from those on the other side, measured in units of each side's own internal variability and weighted toward the statistics that the scribbled classes actually differ on. That map then supplies the boundary supervision that scribbles lack. It trains an auxiliary boundary branch that would otherwise need dense masks, it replaces intensity similarity in label propagation, and it controls where the network is pushed to make sharp, confident decisions. The claim is narrow and testable. Under scribble supervision, gains should appear specifically at boundaries whose texture contrast exceeds their intensity contrast, and gains should be near zero where intensity edges already suffice.

Core Claim

Scribble-supervised segmentation fails at low-contrast boundaries because of the loss, not because of the network's architecture. No term in the objective says where a boundary should lie, so propagation methods and entropy penalties place it wherever intensity or chance dictates. A per-pixel two-sided texture test can fill that role. Its statistics are normalized by within-region dispersion and projected onto the directions that separate scribbled foreground from scribbled background. We expect it to produce better boundaries (lower HD95, higher boundary F-score) than intensity-based propagation, generic edge detectors, or foundation-model pseudo-labels. We also expect the improvement to concentrate where texture, not brightness, distinguishes the tissues.

Where This Sits in Prior Work

Three texture results supply the method. A Statistical Approach to Snakes for Bimodal and Trimodal Imagery evolves a contour to maximize the variance difference between its inside and outside, and so finds boundaries that gradient operators cannot see. We take its core signal, a difference in statistical spread, but compute it locally and without curve evolution, which avoids its local-minimum problems. Significantly Different Textures shows that differences in mean features are masked by high internal variance and must be normalized by dispersion. This is why our statistic divides by within-side spread instead of using raw differences. Cartoon+Texture Image Decomposition shows that under low-pass filtering, oscillating texture loses local total variation quickly while true geometric edges keep it. We use that relative-reduction ratio as one of the local features, so that fine internal grain is not mistaken for a boundary. Lacunarity descriptors (Lacunarity Pooling Layers) are an optional multiscale feature. None of these works uses labels, and none has been connected to weak supervision.

A second group motivates the proposal by showing where boundary handling currently breaks. Learning to Segment Medical Images with Scribble-Supervision Alone expands scribbles by random walk, refines them with a conditional random field (CRF, a pixel-graph model that smooths labels between similar-looking neighbours) on intensity kernels, and masks out uncertain pixels. The research file notes that this pipeline is sensitive to its settings and that it bleeds across low-contrast interfaces. Boundary Perception Guidance pulls scribble predictions toward an off-the-shelf edge detector and nearly matches full supervision on natural images. However, it depends on step-like edges that soft-tissue boundaries lack. MixPolyp sharpens unlabeled regions by minimizing entropy everywhere. The research file notes the tension that this can place a crisp border at an arbitrary location when contrast is low. Our evidence map is meant to decide where such sharpening is justified.

A third group are the direct competitors. MISNet uses a boundary-attention branch trained with edge labels drawn from dense masks. We keep the branch but replace its dense edge targets with the evidence map. SAM-CLNet generates polyp pseudo-labels by prompting the Segment Anything Model (SAM) with scribble-derived boxes. It discards outputs that disagree with the scribbles below a threshold, which the file describes as a response to wandering boundaries on flat polyps. Shape-driven scribble methods add global plausibility through different routes. Adversarial Attention Gates learns a shape prior from unpaired masks. CycleMix uses mixup with connected-component constraints. ShapePU casts the problem as positive-unlabeled learning with cutout consistency. All three are silent about the local statistics at a boundary, so our signal is complementary to them rather than a replacement. DyCON softens consistency penalties on high-entropy boundary pixels and mines hard contrastive pairs from network features, but it needs dense labels for its labeled subset. In our design, uncertainty comes from a statistic of the image rather than from the network's own entropy.

Research Program

At each pixel $x$, for orientation $\theta$ and radius $r$, we split a disk into two halves $A$ and $B$. In each half we compute a small vector of local features $f$: mean intensity or colour, local variance, the local-total-variation reduction ratio across two filter scales, and optionally lacunarity. Let $\mu_A$ and $\mu_B$ be the half-disk means of $f$, and $\Sigma_A$ and $\Sigma_B$ their within-half covariances. The boundary evidence is

$$E(x) = \max_{\theta, r} \; \frac{\left(w^\top(\mu_A - \mu_B)\right)^2}{w^\top(\Sigma_A + \Sigma_B)\,w},$$

where $w$ is a Fisher discriminant direction fitted on the local-feature vectors of scribbled foreground versus scribbled background pixels. This ratio is a dispersion-normalized two-sample statistic of the kind the pre-attentive texture model calls for. The fitted direction $w$ makes it class-specific: if two tissues differ in variance but not brightness, $w$ emphasizes variance. Taking the maximum over $r$ selects the window scale per pixel, which addresses the window-size problem that the research file lists as open for texture segmentation. The ratio is computed once per image, costs a few convolutions, and is converted to a calibrated map by comparing it with its distribution inside scribbled regions, where no boundary exists by construction.

The map enters training in three places, each of which will be ablated separately. First, it defines the pairwise affinity in random-walk scribble expansion: neighbours are separated when $E$ is high, rather than when intensities differ. Second, it serves as a soft target, restricted to pixels at least a margin away from any scribble, for an auxiliary boundary branch of the kind MISNet uses. Third, it gates the entropy-minimization term: sharpening is encouraged only where $E$ is high, and elsewhere unlabeled pixels are left soft. This addresses the MixPolyp tension. In the main version, $w$ is fitted from scribbles alone. A variant that refits $w$ on the network's confident predictions after each round will be tested separately, because it risks the model reinforcing its own mistakes.

Baselines

The naive reference is the same backbone trained with partial cross-entropy on scribble pixels only. For cardiac data, the research file reports an ACDC UNet scribble baseline of 76.6% Dice and an MSCMRseg baseline of 37.8% Dice, both in the CycleMix paper. We will rerun this baseline under our own splits, not quote those figures.

The strongest competitor on polyps is SAM-CLNet, because it is the only method in the file that attacks the same combination of scribble labels and concealed, low-contrast polyps. It reports 0.857 mDice on Kvasir-SEG, against 0.842 for its base CEA-Net (SAM-CLNet). For ETIS it reports only a relative gain of 6.4% over the base model; no absolute number for ETIS was found on Lacuna. We will run it on the same scribble set with the same backbone, give both methods the same training epochs and the same number of hyperparameter trials (the file notes that SAM-CLNet's online training is expensive), and report GPU-hours alongside accuracy.

On cardiac data the strongest scribble competitors are ShapePU, which reports 85.2% average Dice on MSCMRseg (ShapePU), and the random-walk plus CRF pipeline, which reports 88.0% Dice on ACDC and 78.1% on NCI-ISBI prostate (Scribble-Supervision Alone). These figures come from different protocols, so we will retrain every method under a single split and a single tuning budget. Beating a competitor means lower HD95 and higher boundary F-score at matched or better Dice, with significance tested by a paired test across test cases. Fully supervised MISNet, at 0.903 mDice on Kvasir-SEG (MISNet), is reported as a ceiling, not a target.

Four ablations isolate the key ingredient. The first replaces $E$ with plain intensity-gradient magnitude, keeping the three training uses unchanged. The second drops dispersion normalization and uses the raw mean difference. The third replaces the fitted direction $w$ with a uniform weighting, to test whether class-specific adaptation matters. The fourth feeds the same texture feature maps as extra input channels with no change to the loss.

Why not just…?

Why not just run a dense CRF or random walk on raw intensities? This is the method we modify, so the comparison is direct. The authors of the scribble-alone pipeline found its CRF step sensitive to settings and needed uncertainty masking to keep it stable (Scribble-Supervision Alone). Intensity kernels cannot separate regions of equal mean brightness, which is what the variance-differential snake result demonstrates (Statistical Snakes). The first ablation, gradient magnitude in place of $E$ with everything else fixed, measures exactly this. A variant that adds $E$ as an extra CRF kernel will also be run, since it is the cheapest version of the idea and could capture most of the gain.

Why not just prompt SAM or use SAM-CLNet? SAM-CLNet has to reject SAM outputs that disagree with scribbles below $\tau < 0.5$, and its authors describe SAM as lacking domain knowledge, which suggests boundaries are its weak point. It is nonetheless a real threat on Kvasir-SEG, where polyps are often salient. We therefore expect any advantage to appear on ETIS and on the low-intensity-contrast subset of test boundaries, not uniformly. A combination that uses $E$ to score SAM pseudo-labels instead of the scribble-overlap threshold will also be tested.

Why not just use an off-the-shelf edge detector as in BPG? Boundary Perception Guidance was validated only on PASCAL VOC, where boundaries are steps. Colon folds and mucosal grain produce strong responses from generic edge detectors while flat lesion margins do not. We will substitute a pretrained class-agnostic edge map for $E$ in all three training uses. If it matches $E$, the texture statistics are unnecessary and we will say so.

Why not just give the network variance maps as input channels? This is the most serious alternative, because a convolutional network can learn local variance by itself. Our argument is that the weakness of scribble training lies in the missing boundary signal in the loss, not in the network's inputs, and the fourth ablation tests that directly. If input channels alone recover most of the gain, the claim fails.

Why not just use uncertainty-aware consistency as in DyCON? DyCON handles blurry boundaries through the network's own predictive entropy and needs dense labels for its labeled subset. It reports 61.48% Dice on ISLES-2022 with 5% labels, against 53.53% for BCP (DyCON). We will adapt its consistency loss to scribble training as a baseline, and also test gating that loss with $E$ instead of entropy, since image-derived uncertainty and model-derived uncertainty may be complementary.

Proposed Experiments

The primary setting is scribble-supervised polyp segmentation. We train on scribbles following the SAM-CLNet protocol, with SUN-SEG scribble frames as an additional scale test, and evaluate on Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, and ETIS. The secondary setting is cardiac MRI with scribbles, on ACDC and MSCMRseg, using the scribble protocols of CycleMix and ShapePU. Pancreas-NIH with synthetic scribbles generated from its masks tests CT soft tissue.

ACDC is included deliberately as a control. Blood pool and myocardium differ strongly in intensity there, so we predict little gain on ACDC. Large gains on ACDC would suggest that the method works through some mechanism other than texture. Metrics are Dice, HD95, and boundary F-score at a 2-pixel tolerance, together with calibration of predicted probabilities within a band around the true boundary.

The decisive check is a stratified analysis. After training, we use the ground-truth masks to compute, for every boundary segment in the test set, the ratio of dispersion-normalized texture contrast to intensity contrast across it. The claim holds only if the improvement in boundary F-score over the intensity-gradient ablation increases with this ratio and is near zero at low ratios. A flat improvement curve would mean the gains come from generic extra regularization rather than from texture evidence. We will also measure robustness to annotation effort by thinning scribbles to 50% and 25% of their length, since fitting $w$ depends on having enough scribbled pixels.