Segmentation networks for colonoscopy work well on images like those they were trained on. They fail on new hospitals, where polyp margins are fainter, lighting differs, and glare and blur are worse. A standard remedy is domain randomization: train on images whose appearance has been randomly altered by a simulator. The alterations are usually drawn uniformly at random, which leaves gaps in the space of possible conditions. The alternative, adversarial search, keeps hitting the same single failure. We propose to choose the simulated degradations by a rule borrowed from robot exploration. Over training, the share of samples drawn from each region of the degradation space should match a target density, and that density rises where the current model's boundary predictions are worst. This rule, which we call ergodic sampling, guarantees broad coverage and concentrates effort on hard regions at the same time. Unlike most simulation-trained work, every claim is then tested on real, unseen clinical datasets.
The second contribution concerns evaluation. The same degradation simulator gives a cheap, fully controlled stress test. We ask directly whether a model's ranking under simulated degradation predicts its ranking on real external hospitals. If it does not, simulator-only evaluation of boundary robustness is not trustworthy, and the field should know that.
Core Claim
For low-contrast boundary segmentation, robustness to new clinical domains depends on how the space of appearance degradations is covered during training, and not only on how wide that space is. We claim that a sampling rule whose long-run sample distribution matches an error-weighted target density will transfer better to real external polyp datasets than any of three alternatives. The first is uniform pseudo-random sampling. The second is low-discrepancy space-filling sampling, which covers evenly but ignores difficulty. The third is adversarial search, which focuses on difficulty but tends to collapse onto one failure.
The ingredient we expect to matter most is the combination of coverage and focus. The specific choice of space-filling device is not the point. For this reason we deliberately do not use chaotic maps. The ergodicity literature in metaheuristics motivates them, but a Sobol sequence would plausibly do just as well at the coverage half of the problem.
Background and Positioning
Three lines of work motivate the proposal.
On the boundary side, PraNet uses reverse attention, which erases the confident object interior from the network's features so that later layers attend to the ambiguous rim. CaraNet factorizes that attention along image axes to handle small targets. MISNet adds a parallel boundary branch. All three are architectural fixes trained on static real data, and all three still lose substantial accuracy on ETIS-LaribPolypDB. PraNet goes from 0.898 mDice on Kvasir to 0.628 on ETIS. We keep these architectures fixed and change only the training distribution, so our gains would add to theirs rather than compete with them.
On the simulation side, unstructured domain randomization showed that crude variety can beat photorealism for car detection on KITTI (71.5 vs 64.6 AP). Structured domain randomization showed the opposite on fine-grained hard cases, where unconstrained randomization collapsed to 24.0 AP against 52.2 AP for context-aware generation. These two results constrain our design: degradations must respect anatomical context. We therefore apply them only within a band around the annotated polyp margin, and to the surrounding mucosa, rather than scattering random clutter. Neither paper touches low-contrast medical boundaries, and both sample their parameters without any coverage control.
Two papers already search the simulator's parameter space instead of sampling it uniformly. Active Domain Randomization uses a set of interacting particles, steered by a discriminator, over about eight physics parameters for robot control. DeceptionNet trains a network to produce lighting, noise, and distortion perturbations that maximize task error, and reached 32.0% mIoU on SYNTHIA-to-Cityscapes against 22.8% for training on synthetic data alone. Both are the closest prior work. Active Domain Randomization never addresses images or segmentation. DeceptionNet climbs the task loss by gradient ascent, which offers no guarantee that it keeps visiting regions it has not exploited. Our sampler differs from both in an explicit, measurable way: it minimizes the distance between where samples have landed and where they should land.
The sampling rule itself comes from ergodic exploration with binary sensing. That work drives a robot probe so that the time it spends in each region is proportional to a boundary-likelihood map. We keep the ergodic objective but drop the robot dynamics. Our "trajectory" is the sequence of simulator settings, and our "information map" is the segmenter's boundary error.
Method
The degradation simulator has a parameter vector $\theta \in [0,1]^d$, with $d \approx 7$. The parameters control:
- margin contrast, meaning how strongly polyp–mucosa intensity differences are compressed inside a boundary band of controllable width;
- margin blur width;
- illumination falloff and color temperature;
- density and size of specular highlights;
- motion blur;
- sensor noise level;
- a mucosal texture blend.
Each training image is rendered under some $\theta$.
At regular checkpoints we evaluate the current model's boundary error $e(\theta)$ on a held-out slice of the training centers. We measure it as one minus the boundary F-score within a few pixels of the true margin, at a set of probed $\theta$. A kernel regressor smooths these probes into a surface, which defines the target density
$$ \phi(\theta) \propto (1-\alpha) + \alpha \, \hat e(\theta), $$
where $\hat e$ is the normalized smoothed error and $\alpha \in [0,1]$ trades pure coverage ($\alpha = 0$) against pure focus.
The next simulator setting is chosen greedily to minimize the ergodic metric
$$ \mathcal{E} = \sum_{k} \Lambda_k \, |c_k - \phi_k|^2 . $$
Here $c_k$ and $\phi_k$ are the $k$-th cosine-basis coefficients of the empirical distribution of past samples and of the target, respectively. The weights $\Lambda_k = (1+\|k\|^2)^{-(d+1)/2}$ emphasize large-scale mismatch first, following the standard Sobolev-norm form used in ergodic control. With a few coefficients per dimension and $d \approx 7$, this computation is negligible next to training.
When $\alpha = 0$, the method reduces to deterministic space-filling sampling, which gives us a built-in ablation. Mathematically, the rule is closely related to kernel herding toward $\phi$. We present it as an ergodic-coverage sampler because its guarantee is stated in terms of time-averaged coverage, which is the property in question.
Research Program
The work has three parts.
First, we build the boundary-band degradation simulator and check that its marginal statistics are plausible. Margin-contrast histograms of rendered images should overlap those of ETIS and CVC-ColonDB images, computed using their own masks.
Second, we train PraNet and CaraNet under each sampling rule with identical image budgets. We test on real external centers they never saw.
Third, we run the evaluation-validity study. For every trained model, we compute (a) its mean boundary score on a fixed, densely sampled simulated stress grid built from held-out Kvasir images, and (b) its real external mDice. We then report the rank correlation between the two across all models and seeds. This measures, rather than assumes, how far simulation-based evaluation can stand in for real validation in this domain.
We also examine where in $\theta$-space each sampler actually spent its budget. This tests the mode-collapse concern about adversarial search: if the concern is real, adversarial samples should concentrate in a few cells while ours stay spread.
Baselines
The naive reference point is PraNet trained with its published recipe and no degradation simulator. This is standard flips and rotations on the Kvasir-SEG and CVC-ClinicDB training split, tested on ETIS, CVC-ColonDB, and CVC-300. Its reported ETIS Dice is 0.628 (PraNet). We retrain it ourselves under our seeds so that every comparison uses our own numbers. The strongest architecture reported in the file on ETIS is CaraNet, at 74.7% mDice. We also retrain CaraNet with each sampler, because the meaningful question is whether better sampling improves the strongest segmenter, not only the weakest. "Beating" means a gain in ETIS mDice over the same architecture's no-simulator run that exceeds seed-to-seed spread across at least five seeds, without loss on the in-distribution test sets.
Uniform domain randomization draws $\theta$ i.i.d. uniformly, following Tremblay et al.. It is the default practice and isolates whether the simulator alone helps.
A scrambled Sobol sequence over $\theta$ is the strongest coverage-only competitor. It is equivalent in spirit to our $\alpha = 0$ setting and answers whether even coverage is enough.
A DeceptionNet-style adversary uses the same perturbation parameters but updates them by gradient ascent on segmentation loss, following DeceptionNet. It is the strongest focus-only competitor.
An i.i.d. error-proportional sampler draws $\theta$ independently from the same $\phi$ our method targets. This is the key ablation. It has the same target but no coverage control, so it isolates whether the ergodic matching itself, rather than error weighting, produces the benefit.
All samplers get the same number of rendered images, the same training schedule, and the same number of error-probe evaluations; the adversary receives an equal compute budget for its gradient steps. Hyperparameters ($\alpha$, the adversary's step size, the Sobol scrambling seed) are tuned on a held-out portion of the training centers only. External test sets are never touched during tuning.
Why not just…?
Why not just use a Sobol sequence or Latin hypercube instead of an ergodic sampler? This is a real threat to the coverage half of the argument, and it is why we dropped chaotic maps from the design. If the Sobol baseline matches our method, the conclusion is that coverage alone explains the gain and error weighting is unnecessary. Our $\alpha = 0$ versus tuned-$\alpha$ comparison against Sobol settles this directly. The only evidence the file offers in favor of guided over unguided coverage is indirect. Empty-space search found better configurations than random sampling (0.319 vs 0.295 reward), and Active Domain Randomization found that uniform sampling failed in hard regions. Neither compares against low-discrepancy sequences.
Why not just sample in proportion to error, without the ergodic machinery? This is the most dangerous alternative. The two samplers share the same target in the long run and differ only in variance and short-horizon clumping. Our dedicated ablation decides it. If the i.i.d. version is statistically indistinguishable from ours, we will report that the ergodic formulation adds nothing beyond prioritized sampling.
Why not just use adversarial randomization like DeceptionNet? The adversary already targets failure, and it improved SYNTHIA-to-Cityscapes mIoU from 22.8% to 32.0% (DeceptionNet). Our concern is that gradient ascent revisits a single failure mode. The sample-occupancy analysis measures this directly. The external-center results show whether any collapse actually costs accuracy.
Why not just use a stronger architecture or more standard augmentation? Architecture and sampling are orthogonal here, which is why CaraNet appears both as a baseline and as a host for each sampler. The left-atrium study shows that standard flips, rotations, and elastic deformation left thin margins unresolved even as tailored losses raised Dice from 82.01% to 91.54% (Wang et al.). That result motivates targeting the margin specifically, but it does not prove that simulated degradation beats strong standard augmentation. Our no-simulator runs keep the published augmentation recipe so the comparison is fair.
Decisive Checks
Three results would establish the claim.
The first is real external performance. Our sampler should beat both the Sobol and the i.i.d. error-proportional ablations on ETIS and CVC-ColonDB mDice, for both PraNet and CaraNet, beyond seed variance. If only one ablation is beaten, we report which half of the mechanism matters.
The second is coverage behavior. The adversarial baseline should occupy measurably fewer cells of $\theta$-space than ours at equal budget.
The third is the validity of simulated evaluation. The rank correlation between simulated stress-grid scores and real external mDice is reported regardless of its sign. A low correlation would be a substantive negative finding: in-simulator robustness claims for low-contrast boundaries cannot be trusted without real cross-center validation.
References
- PraNet: Parallel Reverse Attention Network for Polyp Segmentation
- CaraNet: context axial reverse attention network for segmentation of small medical objects
- Multi-scale information sharing and selection network with boundary attention for polyp segmentation
- Training Deep Networks With Synthetic Data: Bridging the Reality Gap by Domain Randomization
- Structured Domain Randomization: Bridging the Reality Gap by Context-Aware Synthetic Data
- Active Domain Randomization
- DeceptionNet: Network-Driven Domain Randomization
- Ergodic Exploration Using Binary Sensing for Nonparametric Shape Estimation
- Into the Void: Mapping the Unseen Gaps in High Dimensional Data
- Combating Uncertainty with Novel Losses for Automatic Left Atrium Segmentation