Which Parts of a Rainy, Hazy Scene Can a Restorer Actually Know?

When a camera looks at a scene through rain, fog and sensor noise at once, some of what it has lost can be recovered and some cannot. Rain streaks and grain change from frame to frame, so averaging more frames removes them. Haze behaves differently. Fog that dims a red car looks, in the pixels, much like a duller car in clear air, and no number of extra frames separates the two. Current "all-in-one" restoration networks, and the diffusion models that draw several candidate restorations, give one confident answer either way. They do not tell the user which parts of that answer the data supports and which parts the model filled in from its training set.

This note proposes to use a tool from Bayesian statistics, posterior consistency, as both a test and a design principle for compound restoration. Posterior consistency asks whether the model's distribution over possible answers (its "posterior") shrinks onto the truth as evidence accumulates. Here the evidence is a burst of frames of the same static scene. A correct restorer should become sharper along the directions the frames pin down and stay uncertain along the haze direction. We will build a controlled benchmark that measures both behaviours. We expect existing samplers to collapse along the haze direction when they should not. We will then build a sampler that splits the scene into persistent layers (content, fog transmission, ambient light) and per-frame transient layers (streaks, noise), and we will prove and test that it shrinks at the right rate in the right directions.

Core Claim

Write each frame $t = 1,\dots,n$ as

$$Y_t = \big(X \odot T + A(1-T)\big) \odot (1 - M_t) + S_t + \eta_t,$$

where $X$ is the clean scene, $T = e^{-\beta d}$ is the fog transmission map (depth $d$, density $\beta$), $A$ is the ambient light, $M_t$ is a raindrop occlusion mask, $S_t$ is the rain-streak layer, and $\eta_t$ is sensor noise of level $\sigma$. The hazy-but-clean image $J = X \odot T + A(1-T)$ is the same in every frame. The rain, mask and noise terms are independent across frames. This coupling of streaks with scattering follows the hybrid model argued for in Deep Joint Rain Detection and Removal from a Single Image.

Our claim has two parts.

First, under mild conditions the posterior over $J$ shrinks around the truth at the rate of an averaged-noise problem, roughly $\sigma/\sqrt{n}$ per pixel away from occluded regions. That form comes from the independent per-frame corruptions, which average like repeated measurements.

Second, the posterior over $X$ does not collapse. It converges to the prior restricted to the set of scenes $\{X : X \odot T + A(1-T) = J_0\}$, which we call the haze fibre, and that set has non-zero width.

A sampler that is correct in this sense must show both behaviours. A sampler whose spread along the haze fibre shrinks to zero with $n$, or at fixed $n$ is far narrower than the prior allows, is hallucinating certainty. We predict that single-image conditional samplers fail the second test, and that this failure is the source of the colour-tint errors that LearnIR acknowledges for its learned correction network.

Research Program

Benchmark. The first deliverable is a controllable benchmark, the piece the literature lacks. Existing compound benchmarks such as Outdoor-Rain and the Cityscapes-based Task II of Blind Image Decomposition give one degraded image per scene. Consistency cannot be measured without a growing amount of evidence. We will regenerate those compositions as bursts. Each burst keeps the clean image, depth and fog parameters fixed and resamples streaks, raindrop masks and noise for each frame, with $n$ from 1 to 32 and $\sigma$ swept over the BSD68 noise levels used by PromptIR.

Because the ground-truth layers are known, we can compute, for any sampler, its spread along two kinds of direction. The first is the identifiable direction, meaning changes to $J$. The second is the haze fibre, meaning changes to $X$ that leave $J$ fixed. The fibre spread is computed from the known $T$ and $A$.

We also need a reference for how wide the fibre posterior should be. For small crops and a Gaussian scene prior in a wavelet basis we can compute it exactly. For natural images we estimate it by importance sampling over the fog parameters $(\beta, A)$ under the same diffusion prior the samplers use. The reference is therefore defined relative to a stated prior, not to an unknowable truth.

Sampler. The second deliverable is a factorised posterior sampler. It places a pretrained image diffusion prior on $X$, a smooth low-dimensional prior on depth and fog density, and per-frame streak and noise layers. Each degradation type (rain, raindrop, snow, haze, noise) gets a binary presence indicator with a spike-and-slab prior: the indicator is either exactly off or on with a continuous intensity.

The indicators replace the fixed set of $N$ output heads in BIDeN, whose accuracy falls as $N$ grows. Inference alternates two steps: diffusion-guided updates of $X$ using the explicit per-frame likelihood, and Gibbs-style updates of the indicators and fog parameters.

The design takes its inference strategy from the infinite-dimensional inversion literature surveyed in Infinite-dimensional Bayesian inversion theory and algorithms, which pairs explicit forward models with priors built to keep sampling well behaved at high resolution. That survey assumes a single known physical operator. Here the operator is a composite with unknown parameters.

Theory. The third deliverable is a theoretical result, deliberately narrow. For the indicators we will prove selection consistency, meaning the posterior puts mass tending to one on the true set of active degradations. The proof adapts the discrete Schwartz-type testing argument that Posterior Contraction Rates of the Phylogenetic Indian Buffet Processes used for latent feature sets. It also relies on the mixture-consistency result of Convergence and Loss Bounds for Bayesian Sequence Prediction, which requires the true degradation family to lie inside the model class.

For $J$ we prove the $\sigma/\sqrt{n}$ shrinkage rate away from occlusions. For $X$ we prove convergence to the prior restricted to the haze fibre.

We do not claim neural-network contraction rates of the kind in Posterior Concentration for Sparse Deep Learning. That result covers clean-input regression with bounded ReLU weights and exact posteriors, and it does not transfer to a learned image prior inside an inverse problem. We treat the diffusion prior as fixed and state our guarantees conditional on it, which is where current theory honestly stops. The gap between the exact posterior and our approximate sampler is measured empirically against the small-crop exact reference, not assumed away.

Baselines

Naive reference. The naive reference point is a single-image deterministic restorer applied to frame one, repeated for every $n$. It has zero spread and ignores extra frames, so it anchors the reconstruction-error axis. It is also what most deployed systems do.

Strongest deterministic method. The strongest deterministic competitor is AdaIR, run from its official codebase. It separates degradations by frequency band, putting haze in low frequencies and rain and grain in high frequencies. That matches the persistent-versus-transient split we exploit, so it is the fairest test of whether our physical factorisation adds anything a learned frequency split does not. AdaIR reports 30.53 dB PSNR on SOTS dehazing and a 0.7 dB gain over ArtPromptIR on Rain100L.

For bursts we give it the per-pixel temporal median of the frames as input and retrain it on our burst compositions with the same training images and a matched GPU-hour budget. M2Restore serves as a second deterministic reference, with a reported All-weather average of 31.65 dB PSNR and 0.936 SSIM.

Strongest sampling methods. The strongest sampling competitors are WeatherDiff, a patch-based conditional diffusion model trained on mixed weather, and LearnIR's learned posterior-gradient correction. Both produce distributions, so both can be scored on consistency. WeatherDiff reports no Outdoor-Rain figure that we could find (no reported result found on Lacuna for WeatherDiff's Outdoor-Rain PSNR). LearnIR reports a +2.27 dB PSNR gain on O-HAZE, stated relative to earlier methods rather than as an absolute value.

To give both methods access to multiple frames we condition them on the temporal median, and we also test an extension that averages their guidance across frames. Both are retrained on our compositions under the matched budget.

For decomposition accuracy we compare against BIDeN, which reports 20.07 dB PSNR on its Task I and over 90% accuracy in identifying active layers on Task II.

Ablations. Three ablations isolate our key ingredients. The first removes the persistent/transient split by treating each frame's full degradation as independent. The second removes the presence indicators by keeping every layer always on. The third gives the sampler the true $T$ and $A$ (an oracle), which should make the posterior over $X$ shrink with $n$ like the posterior over $J$. If it does not, the fibre explanation is wrong.

What counts as winning. Beating the baselines means our sampler matches AdaIR-on-median within 0.3 dB PSNR on $J$ while achieving nominal coverage along the haze fibre, where the baselines' coverage should fall well short. Competitive PSNR is necessary but not the win condition.

Why not just…?

Why not just take the temporal median and run AdaIR? For point accuracy this is a serious threat. The median alone removes most streaks and noise, and we expect this baseline to match or beat us in PSNR on $J$ at large $n$. If it also matches us on the scene $X$, then the only thing our sampler adds is calibrated uncertainty, and the note will say so plainly. A deterministic output cannot say how wide the haze fibre is. The oracle-$T$ ablation and the fibre-coverage metric are the tests that decide whether that uncertainty is worth having.

Why not just add deep ensembles or dropout to PromptIR for uncertainty? This is cheap and gives per-pixel spread. However, every ensemble member learns the same mapping from synthetic haze to a clear image, so we expect them to agree on the same colour guess along the fibre and under-cover systematically there. Evidence from classification that Gaussian-prior and Horseshoe Bayesian networks are poorly calibrated out of distribution (Feature Preserving Shrinkage on Bayesian Neural Networks via the R2D2 Prior) suggests that spread from network weights is a weak substitute for spread from the model of the scene. We will include a five-member ensemble at matched compute and report its fibre coverage directly.

Why not just draw more samples from WeatherDiff? WeatherDiff already produces a distribution, and it may already behave consistently. Our benchmark is exactly the test of that. If its fibre spread matches the reference and its spread on $J$ shrinks with $n$, then the factorised sampler is unnecessary, and the contribution reduces to the benchmark and the theory. Our prediction is the opposite, because its training pairs each hazy input with one clear target, which teaches it to collapse the fibre. But this is an empirical question, and we will settle it before building anything else.

Proposed Experiments

Test 1: shrinkage slope. On synthetic bursts built from Outdoor-Rain, BID Task II and SOTS scenes, we plot each sampler's spread on $J$ and on $X$ against $n$ on log-log axes. The prediction is a slope near $-1/2$ for $J$, and for $X$ a spread that levels off at the reference width. A sampler whose $X$-spread keeps falling, or levels off far below the reference, fails.

Test 2: coverage. We measure how often the true $X$ falls within the sampler's 90% credible region along the fibre, with the target being 90%. A companion calibration check covers $J$.

Test 3: exactness. On small crops with the Gaussian prior we compare each sampler's distribution with the exact posterior using a Wasserstein distance. This measures how much our approximation costs relative to the theorem it is meant to satisfy.

Test 4: selection. We score presence-indicator accuracy against BIDeN as the number of candidate degradation types rises from 3 to 8. Our prediction is that the prior-based indicators stay accurate where fixed-head decoding does not.

Transfer check. We run single-frame inference on RainDS-real images, ranking methods by how their uncertainty maps line up with visible fog, since no ground-truth layers exist there. Real multi-frame compound benchmarks are not in the literature we surveyed, so any real-burst result is exploratory.

Decisive result. The experiment that would make this note wrong is WeatherDiff or the ensemble reaching nominal fibre coverage. The result that would make it right is those baselines under-covering while matching us in PSNR, with our oracle-$T$ ablation recovering $X$-shrinkage exactly as the fibre account predicts.