Dual Exposure Maps as Boundary Priors for Polyp Segmentation

Segmentation networks for colonoscopy most often fail at the edges of polyps, and most often in frames that are badly lit. Shadows deep in the colon and glare from the scope's light create strong intensity edges that have nothing to do with tissue. Faint mucosal margins also disappear when the region is dark. This proposal starts from an old observation in image restoration: an underexposed image, inverted ($1 - I$), behaves mathematically like a foggy image. The same holds for an overexposed image without inversion. The standard fog equation can therefore estimate, cheaply and in closed form, how much of each pixel's brightness comes from lighting rather than from the scene. Prior work uses this trick only to brighten images. We propose to use the two lighting maps it produces, one for shadow and one for glare, as a spatial prior that tells the boundary-refinement stage of a segmentation network which edges lighting can explain and where the network's first guess is likely to be unreliable.

The claim is testable before any architecture is built. If boundary errors of strong polyp segmenters cluster where these lighting maps are extreme or sharply varying, conditioning boundary refinement on the maps should cut boundary error on hard unseen datasets such as ETIS. It should do so more than enhancing the image first, augmenting brightness at training time, or using the network's own uncertainty. If the errors do not cluster there, the idea is wrong, and the diagnostic study will show it cheaply.

Core Claim

The exposure inversion duality has a precise form. On the Duality Between Retinex and Image Dehazing shows that dehazing equals one minus Retinex enhancement applied to the inverted image. Retinex enhancement is the family of methods that separate an image into illumination and surface reflectance. LIME makes the related observation that inverted low-light images look like haze. Write the fog model, due to Koschmieder, as $I = Jt + (1-t)A$, where $I$ is the observed image, $J$ the clean scene, $A$ the ambient light colour and $t(x) \in [0,1]$ the per-pixel transmission, meaning the fraction of scene light that survives. Two cases follow:

  • Applying the model to $1-I$ with $A = 1$ gives $I = J\,t_{\text{dark}}$, so $t_{\text{dark}}$ is an illumination map that is small in shadows.
  • Applying it directly to $I$ with $A = 1$ gives $t_{\text{glare}}$, which is small where glare washes out the scene.

This mirrors the forward and reverse passes of Dual Illumination Estimation for Robust Exposure Correction, which corrects under- and overexposure by estimating illumination on $I$ and on $1-I$.

From these maps we define a per-pixel exposure-explained edge score:

$$e(x) = \frac{\max\big(|\nabla \log t_{\text{dark}}(x)|,\ |\nabla \log t_{\text{glare}}(x)|\big)}{|\nabla \log I(x)| + \epsilon},$$

Here $\nabla$ is the spatial gradient and $\epsilon$ a small constant. A value of $e$ near one means the local intensity edge is accounted for by a change in lighting. A value near zero means the edge comes from the scene, that is, from reflectance or tissue.

The central hypothesis has two parts. Boundary errors concentrate where $e$ is high or where the transmission maps are extreme. Routing this information into the boundary stage of a segmenter fixes those errors in a way that image enhancement does not, because enhancement removes the lighting but discards the knowledge of where the lighting was.

Relation to Prior Work

Two papers come closest, and each motivates the proposal by leaving the same gap.

VBLC adapts driving-scene segmentation to fog and night. It uses an "inverse switch" that inverts night images, dehazes them with a saturation-based transmission estimate and re-inverts them. Separately, it normalizes logits to curb overconfidence. The inversion serves only as image preprocessing, and the boundary handling is a global logit constraint with no spatial notion of where edges are unreliable.

AgentPolyp addresses badly lit colonoscopy directly. A model that matches images to text descriptions (CLIP) and a reinforcement-learning agent choose among seven image filters before a U-Net, which lifts Kvasir-SEG mDice from 0.818 to 0.900. It too is enhance-then-segment, and it uses no physical model of lighting.

Our proposal does not output a brighter image as its main product. It outputs a lighting-attribution map that conditions the segmenter's boundary stage. This difference is testable.

The boundary-refinement literature supplies the architecture we modify:

  • PraNet erases a coarse foreground prediction from features so later layers attend to the margin ("reverse attention"). It reaches 0.898 mDice on Kvasir but only 0.628 on ETIS.
  • Polyper erodes and dilates a coarse mask to separate a confident interior from a "potential boundary" band, then cross-attends the two. Removing that separation costs 4.40 mIoU.
  • MISNet adds parallel axial reverse and boundary attention.

The file names two known weaknesses of these designs, and they constrain ours. The band width in Polyper is a fixed morphological kernel. Reverse attention has nothing to erase when the coarse map misses a polyp hidden in shadow. Both weaknesses are exactly where a lighting map should help: it can widen the band where the coarse mask is likely wrong, and it can flag dark regions where a missed lesion is plausible.

Zero-Shot Single Image Restoration Through Controlled Perturbation of Koschmieder's Model also sets a design constraint. It unifies low-light and haze restoration under the same fog model, but it needs about 10,000 test-time iterations per frame. We therefore commit to closed-form, prior-based transmission estimates computed in milliseconds, in the style of LIME and the dark/bright channel priors of Shi et al., plus edge-preserving smoothing.

Research Program

Stage 1: diagnostic study. We train PraNet and Polyper under the common protocol of training on Kvasir-SEG plus CVC-ClinicDB and testing on unseen sets. On their test predictions we compute $t_{\text{dark}}$, $t_{\text{glare}}$ and $e$, and measure how boundary errors are distributed against them. A boundary error here is a false positive or false negative within a few pixels of the ground-truth contour. The question is whether error density in high-$e$ or extreme-transmission regions exceeds what contrast alone predicts. To answer it, we compare against a control map built from plain local contrast $|\nabla \log I|$. If the lighting maps carry no information beyond plain contrast, we stop.

Stage 2: the model. We build the boundary prior into a Polyper-style segmenter in two places.

  1. Adaptive boundary band. Polyper's fixed erosion and dilation are replaced by a band whose half-width at each contour point grows with coarse-mask uncertainty and with local exposure unreliability. Exposure unreliability is computed from $\min(t_{\text{dark}}, t_{\text{glare}})$ and $e$.
  2. Exposure-conditioned boundary attention. The maps enter the boundary attention as additional keys, so the module can learn to discount edges that lighting explains and to borrow evidence from the confident interior instead.

A third, separately ablated component addresses reverse attention's empty-erase failure. Where the coarse map is empty but $t_{\text{dark}}$ is very low over a sizeable region, the module re-runs coarse decoding on features from the restored image $J$. This gives the network a second look exactly where darkness could hide a lesion.

Ambient light colour. We will estimate $A$ per colour channel rather than assuming white. This choice responds to the duality paper's own warning that a neutral $A$ produces colour casts. Endoscopic scenes are strongly red, so we compare per-channel and neutral $A$ directly.

Speed. Everything in the prior is closed-form and differentiable. The target is 30 or more frames per second on 352×352 inputs, matching the live-video regime that PraNet (about 50 fps) and AgentPolyp (about 33 fps) report.

Baselines

The naive reference point is the plain network with no exposure handling: PraNet and our Polyper reimplementation trained with standard multi-scale inputs. The cited reference numbers are PraNet's 0.628 mDice on ETIS and 0.898 on Kvasir. These numbers come from different training protocols, so every baseline will be retrained by us under one protocol: Kvasir-SEG plus ClinicDB training, identical resolution, epochs and optimizer, and the same hyperparameter search budget per method.

The strongest existing boundary method is Polyper, which reports 86.51% mDice on ETIS. That figure far exceeds PraNet's 0.628 and very likely reflects a different split or training set. We therefore treat the reported number as context and compare against our retrained Polyper. Beating it means a statistically reliable gain in boundary metrics on ETIS and CVC-ColonDB without losing mDice. The boundary metrics are boundary F-score at fixed pixel tolerance and 95% Hausdorff distance, i.e. the 95th-percentile distance between predicted and true contours. The reliability test is paired bootstrap over images.

For ColonDB, no reported result was found on Lacuna for any of the cited methods. Likewise, no reported 95% Hausdorff distance on any polyp benchmark was found on Lacuna. These baselines will be established by us.

MISNet (0.903 Kvasir, 0.918 ClinicDB mDice) will be included as a second strong architectural competitor under the same protocol.

The strongest enhance-then-segment competitor is AgentPolyp (0.900 mDice on Kvasir-SEG). Its reinforcement-learning pipeline is costly to reproduce. If a faithful reproduction proves infeasible, we will compare against its reported Kvasir number and add an enhance-then-segment baseline we control: Polyper fed images restored by the same transmission maps our method uses.

Two ablations isolate the key ingredient:

  • The maps concatenated as extra input channels to unmodified Polyper. This tests whether where the prior enters matters.
  • The adaptive band driven by the network's own predictive entropy instead of the lighting maps. This tests whether the physics adds anything beyond self-uncertainty.

Why not just…?

Why not just enhance the frame first and segment the result? This is the most natural alternative and a real threat. AgentPolyp's gain over U-Net shows that enhancement aimed at segmentation helps. Dual Illumination Estimation shows that two-sided correction avoids blowing out glare. Our position is that enhancement erases the information about where lighting was, and the boundary stage needs that information. The controlled baseline that feeds Polyper the restored image computed from the very same transmission maps settles this cleanly. If it matches our method on ETIS boundary F-score and Hausdorff distance, the conditioning claim fails, and the honest conclusion is that physics-based enhancement suffices.

Why not just add brightness, contrast and gamma augmentation, or even random inversion? Augmentation makes the network tolerant of global exposure shifts. The failures we target, however, are spatially local: a shadowed fold next to a polyp, or a specular spot on its rim. VBLC reports that augmentation-style training on night scenes left overconfident boundary errors that its inversion module and logit constraint reduced. That is driving data, though, not endoscopy. Our protocol therefore gives every baseline a strong photometric augmentation suite that includes random local gamma, synthetic vignetting and random $1-I$ inversion. Any gain we report is measured on top of it.

Why not just widen the boundary band using the network's own uncertainty? This is the second real threat, because uncertainty can be high in exactly the badly lit regions. The entropy-driven band ablation tests it directly. The physics prior has one property self-uncertainty lacks: it is available before the network commits to a coarse mask. It should therefore matter most in the empty-erase case, where a lesion missed in shadow produces confident background, not uncertainty. We will report results separately on ETIS images where the baseline coarse map misses the polyp entirely.

Why not just add an edge loss? Niu et al. reduce the 95% Hausdorff distance on spleen CT from 7.57 to 5.88 with a Canny-supervised contour branch, but only together with multi-scale feature aggregation. A Canny edge map also cannot distinguish lighting edges from tissue edges, which is the distinction we are after. We include a contour-loss variant of Polyper as a baseline.

Proposed Experiments

Main comparison. We train on Kvasir-SEG plus CVC-ClinicDB and test on ETIS-LaribPolypDB, CVC-ColonDB and CVC-300. Reported metrics are mDice, mIoU, boundary F-score, 95% Hausdorff distance and frames per second. Every test image is also stratified by exposure statistics, namely the fraction of pixels with very low $t_{\text{dark}}$ or $t_{\text{glare}}$. The predicted signature is specific: gains concentrated in the worst-lit quartile and near zero on well-lit frames. A uniform gain would suggest the method is just adding capacity.

Controlled perturbation. On the Kvasir-SEG test set we apply synthetic local underexposure and specular highlights at graded strength, then plot boundary F-score against strength for all methods. Our method should degrade more slowly than enhance-then-segment and augmentation baselines.

Negative control. We run the identical pipeline on spleen CT. CT intensities are calibrated and carry no illumination, so the prior should yield no gain. A gain there would indicate a capacity effect rather than the mechanism we claim.

Design checks. We compare per-channel against neutral ambient light $A$ on endoscopy, and measure the empty-erase rescue component on the subset of frames where baseline coarse maps miss the lesion.

Decisive outcome. The proposal succeeds if three things hold: the Stage 1 diagnostic shows lighting maps predict boundary errors beyond plain contrast; the full model beats retrained Polyper and the restored-image baseline on ETIS boundary metrics at 30 or more frames per second; and the entropy-band ablation falls short. If the restored-image baseline matches us, the lighting-map conditioning claim fails, and the honest conclusion is that physics-based enhancement suffices.