Underwater photographs are damaged unevenly. Water absorbs red light within a few meters, and suspended particles scatter light back into the lens as a hazy veil. How much of each happens depends on how far each scene point is from the camera. A coral head half a meter away may need almost no correction, while the reef wall behind it has lost its red channel and sits under a thick veil. Current restoration networks still spend the same computation on every pixel. The underwater systems that already use conditional computation, meaning networks that run only a subset of their parameters for each input region, decide which regions matter from learned feature statistics alone. This proposal asks one question: if a restoration network may run its heavy components on only part of an image, should a cheap physical estimate of how degraded each region is decide which part?
The core claim is that the standard way sparse vision models choose which image patches to process gets underwater imagery backwards, and that this can be fixed. In a sparse mixture of experts (a layer with many small sub-networks, where a learned gate sends each image patch to one or two of them and patches beyond an expert's capacity are skipped), patches are usually ranked by how confident the gate is. Distant underwater patches are flat, low-contrast and blue-green. They look like uninformative background and get skipped, yet they need the most restoration. We propose ranking patches instead by a physically estimated degradation severity: low transmission, meaning little direct light survives from the scene to the camera, together with strong veiling light. We will test whether this gives better restoration per unit of computation, with the largest gains in the far, heavily degraded regions.
What exists, and what is missing
Two neighbouring lines of work set the scope. UniUIR is the direct intersection. It routes feature tokens through a Mamba-based mixture of experts (Mamba is a linear-time sequence model), conditions on Fourier-domain prompts and a Depth Anything V2 prior, and refines the output with a latent diffusion stage. It reports 25.11 dB PSNR on T90 at roughly 105 ms. Its experts share out the work in a way learned purely from data, and the authors note that it fails in dark deep-sea scenes lit by spotlights. UniUIR uses its sparsity to separate kinds of degradation, not to budget computation, and its router never sees an optical quantity. On the physics side, Du et al.'s physical model-guided framework estimates veiling light, depth and attenuation coefficients with dedicated branches. It trains with a re-degradation loss: the restored image is pushed back through the imaging model and compared with the raw input. It reaches 29.92 dB PSNR and 0.92 SSIM on LSUI, but every patch passes through the same dense pipeline. We borrow its physical estimates and its loss. What we add is the use of those estimates to decide where computation goes.
The mechanism comes from V-MoE, whose Batch Prioritized Routing ranks tokens by the gate's maximum probability and drops low-priority tokens once expert buffers fill. V-MoE showed this lets a model spend less computation on some images than others, but its priority signal was tuned for recognition, where uniform regions really are uninformative. Spatial Mixture-of-Experts shows that routing by physical regime helps when the physics varies across space. It routes by fixed 2D grid position, though, and underwater degradation varies with scene distance, not image coordinate. We route on an estimated distance-like quantity for that reason. The same paper also warns that fine-grained per-pixel routing can be slower in wall-clock time than dense CNNs despite saving floating-point operations (FLOPs). That constrains us to route patch tokens through fixed-capacity buffers and to report measured latency, not FLOPs alone. Models with Conditional Computation Learn Suboptimal Solutions is a second constraint: learned hard gates often settle in poor local minima. This is part of our motivation. An externally supplied priority signal takes the hardest decision, what to skip, away from the learned gate.
Core Claim
We define a per-token severity score from a lightweight estimate of the imaging model $I_\lambda = J_\lambda t_\lambda + A_\lambda(1 - t_\lambda)$. Here $I_\lambda$ is the observed intensity in colour channel $\lambda$, $J_\lambda$ the clean scene, $t_\lambda$ the transmission, and $A_\lambda$ the veiling light. The score is
$$s(x) = 1 - \tfrac{1}{3}\sum_{\lambda \in \{R,G,B\}} \hat t_\lambda(x) \;+\; \gamma\, \lVert \hat A(x) \rVert,$$
where $\hat t_\lambda(x)$ and $\hat A(x)$ are estimated transmission and local veiling light for token $x$, and $\gamma$ is a weight tuned on validation data. Under a per-image budget of $B$ expert slots, tokens are ranked by $s(x)$ rather than by gate confidence. The top-ranked tokens get two experts, the middle ones one, and the lowest ones pass through only the residual path. The learned gate still chooses which experts a token visits, with $\hat t_\lambda$ appended to its input. Physics decides how much computation a token receives, and learning decides what kind.
The transmission estimate starts from the underwater light attenuation prior (ULAP), which predicts relative depth linearly from the gap between the brighter of the green and blue channels and the red channel. It costs almost nothing and uses no terrestrial depth model. ULAP breaks near artificial lights, where red is locally strong. We therefore refine it with a small convolutional head trained under Du et al.'s re-degradation loss, which also yields the local veiling-light map needed for spotlight hotspots.
Research Program
The backbone is a hierarchical restoration transformer in which the feed-forward block at every other stage is replaced by a mixture of eight experts, following GShard's finding that alternating sparse and dense layers is stable (GShard). The dense and sparse layers keep full resolution, so skipped tokens are still restored by the shared path. Only their extra capacity is withheld. The budget $B$ is expressed as a fraction of tokens and varied from 25% to 100%. This produces a curve of quality against computation for each priority rule, not a single operating point.
The analysis has three parts. First, we measure how well each priority rule matches an oracle: the per-token reconstruction error of a fully dense model of equal active width, which tells us which tokens actually benefit from extra capacity. Second, we test whether $\hat t$ tracks real distance, using the range-calibrated Sea-thru images. Third, we check whether expert assignments line up with physical regimes such as near versus far and hotspot versus shadow, or with semantic content. This addresses the concern that routers may learn to exploit artefacts in the pseudo-references rather than physics.
Baselines
The naive reference point is the same backbone run densely at full computation, together with the same backbone at matched active FLOPs with no experts at all. The first shows what dropping tokens costs. The second shows whether sparsity helps anything beyond parameter count. Both use identical data, training schedule and hyperparameter search budget.
The strongest external competitor is UniUIR, which reports 25.11 dB PSNR on T90 and about 105 ms inference. It is the only underwater mixture-of-experts system in the file, and it holds the best T90 number there. We will retrain it from released code on our split if possible and otherwise report its published T90 figure. Beating it means matching its T90 PSNR within a small margin at under half its FLOPs, or exceeding it at equal FLOPs.
For LSUI we compare against Du et al., reported at 29.92 dB PSNR and 0.92 SSIM. It is the strongest physics-grounded dense model and supplies our estimator's loss. For UIEB we compare against SUCode, reported at 23.857 dB PSNR, 0.925 SSIM and 0.124 LPIPS. CPLDP stands in for soft conditioning, but no exact UIEB number for it was found on Lacuna, so it will be retrained.
The ablation that isolates the key ingredient keeps the architecture, budget and training fixed and changes only the priority rule. The candidates are V-MoE confidence priority, random priority, uniform one-expert-per-token, raw ULAP severity, refined physical severity, and the oracle error-based priority as an upper bound. If refined physical severity does not close a meaningful share of the gap between confidence priority and the oracle, the claim fails.
Why not just…?
Why not just feed the transmission map to a dense network as an extra input or modulation signal? This is the most serious threat. Du et al. and CPLDP already show that dense networks conditioned on physics or on degradation cues work well. If a dense model with $\hat t$ injected through feature-wise modulation, at the same active FLOPs, matches our model, then sparsity adds nothing and the paper becomes a conditioning result. The matched-FLOPs dense-plus-physics baseline is therefore required, and we will report it honestly. Our expectation rests on the fact that modulation cannot move computation between regions. Its advantage should appear at low budgets and in far regions, and the experiments will show whether it does.
Why not just use UniUIR-style learned routing and let the gate discover severity? The gate may learn it. We cannot rule that out from the file, and Gap 3 is an argument, not a measurement. The priority ablation settles the question directly. If confidence priority already tracks the oracle as well as physical severity does, our contribution shrinks to interpretability and we will say so. The suboptimality result for conditional computation gives some reason to expect learned gates to do worse on skip decisions.
Why not just partition the image into fixed depth bins, one expert per bin, with no learning? This is Spatial Mixture-of-Experts routing with depth replacing grid position, and it is cheap. We include it as a baseline in which experts are assigned by quantized $\hat t$. We expect it to lose where a single depth hides different problems, such as a spotlight hotspot and a dark shadow at the same range. The OceanDark and hotspot-region comparisons test that expectation.
Proposed Experiments
We train on the UIEB reference split and LSUI. We test on T90, the LSUI test set and the UIEB test split with PSNR, SSIM, LPIPS and CIEDE2000. Non-reference scores on U45 and the 60 UIEB challenging images are reported only as secondary evidence, since the UIEB authors showed UCIQE and UIQM reward oversaturation. To measure quality by region, which is the heart of the claim, we build a synthetic set with known per-pixel transmission by applying the imaging model to terrestrial RGB-D images, following Hu et al. and WaterGAN. We then report PSNR separately for tokens in the lowest-transmission third. Sea-thru supplies the check between estimated transmission and true range, and OceanDark the qualitative spotlight test. Latency is measured on a single embedded-class GPU and on a desktop GPU, with the dense model timed on the same hardware.
The decisive check is whether the curve of quality against computation for refined physical priority lies above confidence priority and random priority, with the gap concentrated in low-transmission regions. At a 50% budget it should also stay close to the full-computation model. A second check is that measured latency, not just FLOPs, falls as the budget falls, which the Spatial Mixture-of-Experts wall-clock warning makes a real risk. If physical priority wins only on FLOPs and loses on latency, or if the dense-plus-physics baseline matches it at equal computation, the proposal's claim is falsified.
References
- UniUIR: Considering Underwater Image Restoration as An All-in-One Learner
- A Physical Model-Guided Framework for Underwater Image Enhancement and Depth Estimation
- Scaling Vision with Sparse Mixture of Experts
- Spatial Mixture-of-Experts
- Models with Conditional Computation Learn Suboptimal Solutions
- A Rapid Scene Depth Estimation Model Based on Underwater Light Attenuation Prior for Underwater Image Restoration
- GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- Enhancing Underwater Images via Adaptive Semantic-Aware Codebook Learning
- Conditional Prompt Learning via Degradation Perception for Underwater Image Enhancement
- An Underwater Image Enhancement Benchmark Dataset and Beyond
- Underwater Image Restoration Based on Convolutional Neural Network
- WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images