Measuring Edges in Units of Noise: Sparse-Label Segmentation of Brightened Dark Images

Many medical and robotic images are taken in the dark, for example the far walls of an endoscopic cavity or a night-time scene. Two things are then true at once. First, a person has to brighten the image before anyone, human or model, can see anything. Second, experts can afford to mark only a few points or brushstrokes (scribbles) per image. The standard way to turn those sparse marks into full masks is label propagation. It spreads each mark outward to neighboring pixels until it hits an intensity edge. Methods of this kind include random walks, conditional random fields (CRFs), and neighborhood-consistency losses. They all decide what counts as an edge with one global threshold on intensity difference. Brightening breaks that assumption. It multiplies sensor noise by a gain that is large in shadows and small in highlights. As a result, noise in dark regions looks exactly like an edge, and propagated labels either stop at phantom boundaries or leak through real ones.

This note proposes a simple fix: measure every intensity difference in units of the local noise that the enhancer left behind. For any enhancer, we can estimate a per-pixel map of that residual noise. For a brightening curve, the estimate is analytic. For an enhancer that also denoises, we estimate it by pushing small noise perturbations through the network and measuring how much the output varies. Dividing pairwise differences by this map makes the propagation's edge test statistically comparable across bright and dark regions. The method needs no normal-light reference images, no paired data, and no retraining of the enhancer. It can be inserted into existing scribble and point pipelines.

Core Claim

Sparse-label expansion fails in brightened dark images mainly because the affinity (the similarity score between neighboring pixels that governs how labels spread) is miscalibrated, not because features are missing. The enhancer itself tells us how much it amplified noise at each pixel. Using that to normalize affinities should recover most of the gap between propagation on well-lit images and propagation on brightened dark ones. The recovery should grow with how dark the image was.

Concretely, let $I$ be the dark input and $E(I)$ the brightened output. Let $\sigma^2_{\text{in}}(x)$ be the input noise variance at pixel $x$, fitted per image with the standard signal-dependent model $a\,I(x)+b$, which accounts for photon shot noise plus read noise. We estimate the output noise variance as

$$\hat\sigma^2(x) = \mathrm{Var}_{\epsilon}\big[E(I+\epsilon)(x)\big], \quad \epsilon \sim \mathcal{N}\big(0, \mathrm{diag}(\sigma^2_{\text{in}})\big),$$

where the variance is taken over a few dozen perturbed forward passes. For a pointwise curve such as Zero-DCE, this reduces to the first-order formula $\hat\sigma^2 = (\partial E/\partial I)^2\sigma^2_{\text{in}}$. The propagation affinity between neighboring pixels $i$ and $j$ then becomes

$$w_{ij} = \exp\!\left(-\frac{\lVert E(I)_i - E(I)_j\rVert^2}{2(\hat\sigma^2_i + \hat\sigma^2_j) + \beta}\right),$$

where $\beta$ is a small floor that keeps well-lit regions from becoming hypersensitive. The standard affinity replaces the denominator with a single global bandwidth. The proposal is exactly this substitution, plus a matching reweighting in any learned affinity loss.

Relation to Prior Work

The motivating evidence comes from the enhancement literature. CERL (CERL) jointly enhances and denoises under realistic intensity-dependent noise. It documents that brightening first turns sensor noise into complex non-Gaussian patterns, and that denoising first erases faint edges. Two task-oriented studies report that enhancement artifacts can hurt downstream models more than darkness itself. The first is a dual-teacher low-light pose estimator (Ai et al.), which adapts pose estimation to extreme darkness. The second is an infrared odometry method (Passive Infrared Image Detail Enhancement), which uses gain masks to keep amplified noise from becoming false features. SCT (Tracker Meets Night) adds an explicit noise term to its curve for tracking. All of these fix the image or the features. None asks what amplified noise does to the pairwise affinities that label propagation depends on.

The weak-supervision side supplies the methods we modify. The scribble-only method (Learning to Segment Medical Images with Scribble-Supervision Alone) expands scribbles with a random walk and refines them with a CRF, using dropout-based uncertainty to hold back ambiguous pixels. It reaches 88.0% Dice on ACDC. PNL (Point-Neighborhood Learning) grows single points on endoscopy images into circular neighborhoods inside a mean-teacher loop. ProCNS (ProCNS) aligns features with class prototypes through a regional affinity loss and masks regions where predictions disagree. It reaches 82.73% Dice on Kvasir-SEG. All three were developed on reasonably exposed images with a global notion of edge strength. ProCNS's noise masking targets semantic disagreement, not physical noise. We keep their machinery and change only how edges are measured.

The closest work is DGKD-WLSS (Diffusion-Guided Knowledge Distillation). It is the only weakly supervised low-light segmenter in the file and reaches 46.3% mIoU on the real-world LIS dataset from image-level tags. It avoids brightening altogether. Instead, it distills from a teacher trained on normal-light images and uses a natural-image depth model as a structural prior. Both ingredients are unavailable inside a body cavity. Our proposal differs in three ways. It accepts brightening instead of avoiding it. It works on sparse spatial marks rather than image tags. And it needs no normal-light teacher or depth model, only the enhancer and a per-image noise fit.

Research Program

The first step is diagnostic. We will show that affinity miscalibration is the failure mode, rather than assuming it. We take images with dense ground truth, darken them with a calibrated noise model, brighten them with several enhancers, and run standard propagation from simulated scribbles and points. We then measure where propagated labels cross true boundaries or stop early. The hypothesis predicts that errors concentrate where $\hat\sigma$ is high, independent of the true local contrast. If errors instead track low true contrast regardless of noise, the premise is wrong and we stop.

The second step builds the noise-map estimator for three kinds of enhancer. The first kind is pure curves: Zero-DCE (Zero-DCE), whose authors note it has no denoising, so its gain is analytic. The second is a curve with embedded denoising: CLODE (CLODE). The third is joint enhancement and denoising: CERL. For the second and third, residual noise is spatially complex, and the resampling estimator is required. We validate the estimated maps against the true output noise in synthetic settings, where it is known exactly.

The third step inserts noise-normalized affinities into three pipelines: the random walk and CRF of the scribble method, PNL's neighborhood supervision, and ProCNS's regional affinity loss. For the learned losses, each pairwise term is weighted by the same noise-normalized affinity. No other hyperparameters change.

Baselines

The do-nothing reference is standard propagation on the dark input with no brightening. The second reference is naive enhance-then-propagate: the same pipeline run on Zero-DCE, CLODE and CERL outputs with global-bandwidth affinities. DGKD-WLSS already shows that this route underperforms on LIS, where Zero-DCE plus segmentation reaches 40.2% mIoU and EnlightenGAN reaches 38.9%. We expect the same ordering with sparse labels.

The strongest sparse-label competitors are ProCNS and PNL, run on the best enhancer's output with their published settings. To keep the comparison fair, each receives the same seeds, backbone, training schedule and enhancer, and the same hyperparameter search budget as our variant. That budget includes tuning the global affinity bandwidth, so that a well-tuned single bandwidth cannot be the whole story. On dark endoscopy with sparse labels, no reported result was found on Lacuna. ProCNS's 82.73% Dice on well-lit Kvasir-SEG serves as an upper reference. Beating these competitors means a statistically significant Dice gain over three seeds at every darkness level below well-lit, with no loss at the well-lit level.

On LIS, DGKD-WLSS's 46.3% mIoU is reported as a reference, not a head-to-head comparison. It uses image tags, which carry less information than our points. The fair test is whether noise-normalized propagation matches or exceeds it with the fewest points per object.

Three ablations isolate the key ingredient. The first uses an oracle noise map in the synthetic setting, which bounds how much estimation error costs. The second uses only the enhancer's gain, ignoring its denoising, which tests whether resampling matters for denoising enhancers. The third uses a spatially shuffled noise map with the same histogram. That control checks whether gains come from matching noise to location, rather than from a generic change in bandwidth.

Why not just…?

Why not just use a better joint denoising enhancer and keep standard propagation? This is the most serious threat. If CERL's output were clean enough, a global bandwidth would suffice. CERL's own analysis shows residual noise remains spatially variant, and DGKD-WLSS shows enhanced images still lose to feature-level approaches. That is suggestive, not conclusive. The CERL-plus-tuned-ProCNS baseline decides the question. If our gain over it vanishes, the honest conclusion is that joint denoising already solves the problem, and our contribution shrinks to the diagnostic.

Why not just compute affinities on the dark input after a variance-stabilizing transform? For a pointwise curve, this is nearly equivalent to our method, and we say so. The transform whitens the input noise, and brightening only rescales it. The difference appears with enhancers that denoise or mix pixels spatially, where residual noise is no longer the input noise times a gain. It also appears with processed, compressed endoscopy video, where a clean raw noise model is unavailable. We include the transform as a baseline. We expect it to match us with Zero-DCE and fall behind with CLODE and CERL. If it does not fall behind, we will recommend it as the simpler recipe.

Why not just prompt a segmentation foundation model with the expert's points? Active Label Correction (ALC) found that Grounded-SAM masks on Kvasir-SEG are noisy enough to need expert correction even on well-lit frames. Amplified noise should make fragmentation worse. We run SAM-style point prompting on the same seeds and enhanced images as a baseline. We also test whether our noise-normalized CRF improves its masks, because the two approaches are compatible.

Why not just train with dark-plus-Gaussian-noise augmentation? R2RNet (R2RNet) reports that synthetic Gaussian darkening does not transfer to real sensor noise. The pose study needed more than augmentation to close its gap. Augmentation also does not touch the propagation step, which is not learned in the random-walk pipeline. We include an augmentation-trained ProCNS as a baseline.

Proposed Experiments

The synthetic-dark benchmarks are Kvasir-SEG and NPC-LES endoscopy. We darken each image in an approximately linearized intensity space to several mean-intensity levels, down to the extreme regime of about 2/255 studied by Ai et al. We add Poisson-Gaussian noise with parameters in a realistic range, then re-apply a tone curve. We simulate scribbles and single points per lesion from the dense masks.

The real-noise benchmark is LIS, whose real low-light images and dense masks allow the same simulated sparse annotations. As a stress test with naturally dark regions, we also stratify real Kvasir-SEG frames by local brightness and report Dice separately for the darkest band.

The metrics are Dice and mIoU. We add boundary-leak rate, the fraction of propagated pixels outside the true mask, and early-stop rate, the fraction of true-mask pixels unreached by propagation. To speak directly to the annotation bottleneck, we report annotation efficiency: the number of points per lesion needed to reach 90% of fully supervised Dice.

Three results would be decisive. The first is that our Dice gain over the strongest baseline grows monotonically as images get darker and is near zero on well-lit images. The second is that the shuffled-map control loses most of the gain. The third is that, on LIS, the method matches DGKD-WLSS's 46.3% mIoU with a small number of points per object and no normal-light teacher. If only the first holds with Zero-DCE, and the variance-stabilizing baseline matches us there, the contribution reduces to showing that noise-aware propagation matters in brightened dark images. We will report it as such.