Teaching a Fast Segmenter Where the Edge Might Be: Interior-Anchored Band Labels for Self-Training

Clinical video segmentation, such as outlining a flat polyp during a live colonoscopy, needs two things that pull against each other. The model must run at video rate (25–60 frames per second) on modest hardware, and it must place an edge where the lesion fades into similar-looking tissue with no sharp change in brightness. Methods that handle such faint edges well rely on extra network branches, repeated stochastic forward passes, or large ensembles. Methods that run fast drop exactly those components. When labels are scarce and models train on their own predictions ("self-training with pseudo-labels"), the faint edge is also where errors build up: boundary pixels sit near 50% confidence, and each round of retraining can lock in a wrong guess.

We propose to change what the pseudo-label says at the boundary instead of adding machinery to the deployed network. Each self-training round produces a three-part label anchored on the lesion's confident interior. The interior is marked foreground. Clear background is marked background. Between them lies an "unresolved band" where the label makes only one claim: foreground probability must not increase as one moves away from the interior. Across rounds the band narrows only where the predicted edge position has stopped moving. A lightweight single-pass student trains on these labels, so the costly part of resolving the edge happens entirely during training, and the deployed model is a plain fast segmenter. The aim is a student that keeps video-rate speed and draws better faint edges than the same student trained with hard pseudo-masks, confidence filtering, or standard teacher-to-student distillation.

Core Claim

Existing semi-supervised methods give an ambiguous boundary pixel one of two treatments. They either discard it as noise, as the ensemble quality filter of Auto-Annotation Quality Prediction for Semi-Supervised Learning with Ensembles does with its most dubious 3.8% of pixels, or they keep it with a soft, entropy-scaled penalty, as DUCore / DyCON argues one should. We claim there is a third option that suits low-capacity real-time networks better: supervise the ambiguous region with a constraint on shape instead of a value. The constraint says "the edge lies somewhere in this band, and the mask is monotone across it." That statement is almost always true even when the exact edge position is not known. It therefore carries real information without hardening a wrong boundary, which is the confirmation-bias failure that self-training is known for at fuzzy margins.

The interior anchor comes from two papers that supply the method. Polyper splits a coarse mask into an interior region and a potential-boundary region with fixed erosion and dilation, then uses attention to pass interior features outward. Its ablation shows that removing this region separation costs 4.40 points of mean intersection-over-union, which is why we treat the interior prior as worth using. OP-SAM uses a distance transform to find interior points far from any boundary as stable anchors. We take the anchoring idea from both but use it differently. Polyper applies the prior inside the network at inference time, with fixed kernel widths and no reported speed. OP-SAM applies it to prompt a foundation model. We move the prior into the label and remove it from the deployed architecture.

Research Program

Write $I_t$ for the confident-interior set at self-training round $t$: pixels where the current teacher (an exponential moving average of the student, plus a stronger offline model in the first round) is confident and lies deep inside the predicted region by distance transform. Write $d(x)$ for the distance from pixel $x$ to $I_t$, and $w_t(x)$ for the band width, normalised by the object's own interior depth. Normalising by object size removes the fixed-kernel problem Polyper reports across extreme polyp sizes. Pixels in $I_t$ get a foreground label. Pixels beyond the band get a background label. For neighbouring pixel pairs $(x,y)$ inside the band with $d(x) < d(y)$, the student's foreground probability $p$ pays a monotonicity hinge:

$$L_{\text{band}} = \sum_{(x,y)\in B_t,\ d(x)<d(y)} \max\big(0,\ p(y) - p(x)\big).$$

Together with the hard labels on either side, this forces exactly one inward-to-outward transition inside the band. The student chooses where that transition falls, guided by image evidence. The constraint does not dictate it.

The refinement step updates the band. For each ray from the interior, we record where the student's prediction crosses 0.5 in round $t$ and in round $t-1$. Where the crossing has moved by less than a tolerance, the band shrinks toward it. Where it is still moving, the band stays wide. The round-to-round history therefore acts as a free uncertainty estimate. It replaces the multi-pass Monte Carlo dropout of SAM-MLC and the 18-model ensemble of the quality-filter paper, both of which the file flags as heavy even offline. Following SAM-MLC's finding that medical pseudo-labels fail at the level of whole images, a frame is dropped from the round entirely if its interior set is empty or jumps discontinuously from the previous round.

The student is a single-pass lightweight encoder-decoder with no boundary branch at inference. We use an M2U-Net-style network (M2U-Net, 0.55M parameters) and a MobileNet-scale polyp segmenter. Because the band exists only in the loss, the deployed model costs the same as the plain baseline. Latency becomes a hard constraint that we measure on a desktop GPU and on an embedded ARM board. The embedded measurement matters because M2U-Net's 19.9 ms on a GTX 1080 Ti became roughly 0.6 s on a Rockchip system-on-chip.

Baselines

The naive reference is the same lightweight student trained only on the labelled fraction with cross-entropy plus Dice loss. A second naive reference is that student trained with vanilla Mean-Teacher and a fixed 0.95 confidence threshold, which by construction discards most boundary pixels. Both use the identical architecture, input resolution, augmentation, and number of training iterations as our method. Any gain on these comparisons comes from the labels alone.

The strongest competitor for the question we are actually asking is a two-stage pipeline. First, train a strong semi-supervised boundary-aware model offline. Second, distill it into the same student using the boundary-consistency distillation of Improving Fast Segmentation With Teacher-Student Learning. That paper distills both probability maps and an 8-neighbour logit-difference map, so the student copies the teacher's local edge structure. It also adds teacher pseudo-labels on unlabelled images. On Cityscapes it reports 71.9% mean intersection-over-union at 20.6 frames per second, beating its 7.4 frames-per-second teacher. For the teacher we use LoCo on still images and VPSentry on video. The teacher gets the same labelled split and an offline compute budget at least equal to all our refinement rounds combined. Distillation hyperparameters get the same tuning budget, a fixed number of trials on a held-out labelled validation split, that we give to the band width and tolerance.

We also compare against existing fast semi-supervised systems at their reported operating points. TCCNet reports 0.824 mean Dice on CVC-300 with 1/15 labels at 65 frames per second. VPSentry reports 83.95 Dice on SUN-SEG-Hard with 10% labels at 28–30 frames per second on an RTX 4090. These figures set the accuracy-speed frontier our student must reach or move. Beating the pipeline means one of two things at equal measured latency on the same device: a higher boundary-focused score, or matching region Dice on the Indefinable-Boundaries subset at lower latency. A gain only on overall Dice would not count, because that metric is dominated by object interiors.

Three ablations isolate the key ingredient. The first replaces the monotone band with an ignore band of identical geometry, which is the exclusion philosophy. The second replaces it with an entropy-weighted soft label, which is the exploration philosophy. The third freezes the band width across rounds, so there is no stability-driven narrowing. If the ignore-band ablation matches the full method, the monotonicity idea is not doing the work.

Why not just…?

Why not just distill a strong semi-supervised teacher into the fast student? This is the real threat. If LoCo or VPSentry already places faint edges well, then Xie et al.'s boundary-consistency distillation might carry that skill into the student without any new labelling scheme. Our argument is that a teacher's confident but wrong boundaries get distilled as faithfully as its right ones. The band label is designed to transmit only the part of the edge the teacher is sure about. The direct comparison above settles this question. We also stratify results by teacher boundary error. If our method's advantage does not concentrate on frames where the teacher's edge is wrong, the hypothesis fails and the simpler pipeline should be preferred.

Why not just train the fast student with an existing semi-supervised method like TCCNet, which already runs at 65 frames per second? TCCNet's co-training updates pseudo-labels between neighbouring frames and relies on reverse attention over already-extracted features. The file notes that severe initial mislocalisation then leads to boundary clipping it cannot recover from. TCCNet also never re-estimates its training masks across rounds. Our band labels are architecture-agnostic, so the cleaner test is to apply them to the TCCNet student itself. If TCCNet with band labels does not beat TCCNet as published on CVC-300 and ETIS, the contribution is not useful to the fastest existing systems.

Why not just generate pseudo-labels once with the Segment Anything Model and train the student on them? SAM-MLC reports that zero-shot SAM masks on medical images contain whole-image failures and blurred edges. Training directly on them degrades the downstream network, and the degradation is worst on breast ultrasound. OP-SAM shows that avoiding manual prompts takes iterative prompt evolution of its own. We still run this baseline, SAM masks from OP-SAM-style automatic prompts with a single round of student training, because it is cheap. We also use SAM as an optional first-round interior source for our method. That isolates whether the gain comes from SAM or from the band refinement.

Proposed Experiments

The main still-image study uses Kvasir-SEG and CVC-ClinicDB with 10% labels for training, and ETIS and CVC-ColonDB as unseen test domains. LoCo reports 86.18% mean intersection-over-union on Kvasir-SEG at 10% labels as the reference point for semi-supervised accuracy, without a speed figure. We add FAHSYU-LC laryngeal images, where LoCo reports 62.57% mean intersection-over-union at 10% labels, because its benign-tumour class is the clearest low-contrast case in the file. The video study uses SUN-SEG with 10% labelled frames. We report results separately on the Hard split and on the Indefinable-Boundaries and Small-Objects attribute subsets. The student runs per frame, so this tests whether label quality can stand in for VPSentry's temporal memory at higher speed. EndoVis17 instrument segmentation serves as a check on transfer outside polyps. There, DMNet reports 61.03% mean Dice at 38 frames per second. Its targets are sharp-edged, so we expect little gain, and a large gain would make us suspect a confound.

Metrics are region Dice and intersection-over-union, a boundary F-score within a small pixel tolerance, and 95th-percentile Hausdorff distance. The file contains no reported boundary-metric results for these benchmarks, so we compute all boundary scores ourselves for every baseline. Latency is measured end to end at batch size one, on an RTX-class GPU and on an embedded ARM board, alongside parameter count.

Three checks decide whether the idea holds. First, at equal measured latency, the band-label student must beat the distillation pipeline on boundary F-score on SUN-SEG Indefinable-Boundaries and on ETIS. Second, the monotone band must beat the ignore band and the entropy-weighted band of identical geometry. Third, across rounds, the edge-crossing positions must converge toward expert contours, measured on a held-out labelled subset, rather than drift outward. That third check tests directly the over-segmentation that the file attributes to propagation-based video pseudo-labelling. We also report the band's final width per frame as a byproduct uncertainty map. If its width does not correlate with expert disagreement or boundary error, we will say the method is only a regulariser and not a calibrated statement of where the edge might be.