Learning to remove haze from images runs into the same wall as learning to solve hard optimization problems: the "answer" a network should learn from is the expensive part. For dehazing, the answer is the clear image, and outdoors it cannot be photographed at the same moment as the hazy one. Real haze also makes the problem harder, because it is patchy. Smoke, mist and exhaust form pockets that drift with the wind, so the textbook model of uniform haze breaks down. We propose to treat this drift as a source of supervision instead of a nuisance. For a fixed camera, such as a traffic or surveillance camera, the scene behind the haze barely changes while the haze moves across it. Each background pixel is therefore seen through many different amounts of fog. If the fog is also required to move the way a physical substance moves, that is, carried by wind and spreading out over time, then the depth of the scene and the density of the fog become separately recoverable from unlabeled video.
The concrete proposal has two stages. First, a pair of networks is trained, without any clear images, to solve a constrained physical inverse problem on each video clip. One network predicts the clear background, the depth, and a time-varying fog-density field. The other predicts per-clip penalty weights that enforce the physical constraints, following the self-supervised primal-dual recipe developed for optimization. Second, the resulting physically consistent reconstructions are used as pseudo-labels to distill a fast streaming video dehazer. The open question this addresses is one that current methods leave unsolved. When haze is non-homogeneous, local fog density and scene depth are entangled. Moving-camera methods untangle them using brightness constancy, and that assumption fails exactly where fog is turbulent.
Core Claim
In the standard scattering model, a hazy pixel is $I(x)=J(x)t(x)+A(1-t(x))$. Here $J$ is the clear radiance, $A$ the ambient airlight, and $t=e^{-\beta d}$ the transmission. A single image only determines the product of the scattering density $\beta$ and the depth $d$, and only weakly. Our claim is that for a static camera the factorization $\beta(x,\tau)\,d(x)$ becomes identifiable, up to one global scale, under three conditions. The depth must be held fixed over time. The background radiance must be held fixed over time. The density, projected into the image, must be constrained to obey an advection-diffusion law on a graph of image regions. The intuition is simple. When a fog pocket of a given density drifts from region $i$ to region $j$, the change in observed optical thickness reveals the ratio $d_j/d_i$.
Writing $\rho_i(\tau)$ for the projected density at region (graph node) $i$ and time $\tau$, $v_{ij}(\tau)$ for a learned flow along graph edge $(i,j)$, and $\kappa$ for a diffusion rate, the transport residual is
$$ r_i(\tau)=\rho_i(\tau+1)-\rho_i(\tau)-\sum_{j\in\mathcal N(i)}\big[v_{ji}(\tau)\rho_j(\tau)-v_{ij}(\tau)\rho_i(\tau)\big]-\kappa\sum_{j\in\mathcal N(i)}\big(\rho_j(\tau)-\rho_i(\tau)\big). $$
Each clip then defines the problem of minimizing the reconstruction error $\sum_\tau \|I_\tau - J e^{-\rho_\tau d}-A_\tau(1-e^{-\rho_\tau d})\|^2$ over the background pixels. The minimization is subject to four constraints: $|r_i(\tau)|\le\epsilon$, $\rho\ge 0$, $J\in[0,1]^3$, and a smoothness bound on $A_\tau$.
Projecting 3D wind into the image plane is an approximation. For that reason transport enters as a bounded inequality with tolerance $\epsilon$ rather than as an exact equation. What we claim is new is the use of fog motion itself as the separating signal, enforced as a hard-ish constraint through learned multipliers rather than as a fixed penalty.
Relation to Prior Work
The closest work comes from driving-video dehazing. Driving-Video Dehazing with Non-Aligned Regularization (DVD) avoids paired labels by retrieving non-aligned clear frames from similar routes, and aligns frames with flow-guided and deformable attention. Depth-Centric Dehazing and Depth-Estimation from Real-World Hazy Driving Video (DCL) goes further. It learns a spatially varying $\beta(x)$, couples the scattering model with brightness consistency from camera motion, and regularizes with adversarial discriminators. Both papers motivate our proposal, since they show unpaired video supervision works. Both also constrain it. They rely on camera ego-motion and brightness constancy, which DCL's own setting implies is violated when fog itself moves, and their constraints are adversarial rather than explicit. We invert their assumption. Our camera is static, and the motion we exploit belongs to the fog.
HazeFlow synthesizes non-homogeneous haze maps by simulating Brownian particle diffusion and trains a rectified-flow dehazer on them. It supplies a simulator idea we reuse to build synthetic test clips with known truth, but it is static and models no transport over time. Non-Homogeneous Haze Removal via Bidimensional Graph Reasoning builds graphs over semantically similar pixels to pass clear-region information to hazy regions. It is supervised and single-image, and its graph encodes appearance similarity, not physical transport.
The optimization side supplies the method. Self-Supervised Primal-Dual Learning (PDL) trains a primal network for decisions and a dual network for instance-specific Lagrange multipliers, mimicking an augmented Lagrangian method. It reports optimality gaps of 0.176%–0.324% on QP against 4%–6% for the penalty-style DC3, with zero label-generation cost. We transplant this recipe to a physical inverse problem, where checking a candidate reconstruction is cheap even though finding one is not. Learning for Dynamic Combinatorial Optimization without Training Data shows that naively warm-starting from the previous time window traps unsupervised solvers in poor optima, and proposes shrink-and-perturb re-initialization. That result constrains our streaming design, as described below. Continuous Learned Primal Dual offers adjoint-based memory savings for unrolled physical inverses, which we keep as an option if long clips exhaust memory.
The flow field $v_{ij}(\tau)$ is predicted by a spatio-temporal graph network. We follow GMAN in using spatial and temporal attention over nodes rather than a fixed adjacency, because wind direction changes. We use T-GCN's recurrent graph cell as the cheaper alternative. In those papers graph networks forecast clean sensor values. Here they parameterize a physical constraint.
Research Program
The graph has one node per background superpixel of the static view, computed once per camera from a temporal median frame, with edges between adjacent superpixels. Pixels where the static-background assumption fails are masked out of the background solve. This includes moving vehicles and pedestrians, detected as outliers to the per-pixel scattering line in color space. The reason for the color-space test is that under the scattering model a static pixel's hazy colors over time lie on the segment between $A$ and $J$.
The primal network reads a clip of 32–64 frames. It outputs $J$, $d$, $\rho_\tau$, $A_\tau$, and edge flows $v$. The dual network reads the same clip and outputs multipliers for the transport, positivity and range constraints. Training alternates, following PDL. The primal network minimizes the augmented Lagrangian, the dual network regresses toward the updated multipliers, and the penalty coefficient grows on a schedule. The training corpus is thousands of unlabeled clips from fixed outdoor cameras in fog. For streaming use on a long-running camera, the primal network is re-initialized per window with shrink-and-perturb rather than plain warm-starting. We will test directly whether the warm-start trap reported for combinatorial problems also appears here.
In stage two, the stage-one reconstructions, including the masked foreground, are used as pseudo-labels. The foreground in those targets is filled by applying the recovered $t$ and $A$ to the observed pixels. These targets train a causal video student, a ConvLSTM-style recurrent encoder-decoder that sees only past frames, intended for real-time deployment. We also analyze identifiability on synthetic clips. A haze field with known wind and known depth is advected over clean static scenes, which lets us measure how depth-ratio recovery error falls as fog motion increases. We expect it to fail when fog is static, which is the regime of single-image methods.
Baselines
The naive reference point has two parts. The first is the raw hazy input, scored with no-reference metrics. The second is classical Dark Channel Prior dehazing applied per frame, which HazeFlow also uses as an initializer and notes is unreliable in bright sky.
The strongest existing unpaired video method is DCL. It is the strongest competitor because it already learns spatially varying scattering from real video without aligned pairs, and on GoProHazy it reports a 22.04% NIQE improvement over DVD. That is a relative figure; the file gives no absolute NIQE value. On static-camera footage DCL's ego-motion depth cue vanishes. To keep the comparison fair, we will also fine-tune DCL on our static-camera corpus with its non-aligned regularization, using clear-weather frames from the same cameras as its non-aligned references. It gets the same compute and hyperparameter budget as our method.
The strongest single-image method is HazeFlow, applied per frame. The file reports no NH-HAZE number for it ("no reported result found on Lacuna for HazeFlow on NH-HAZE"). The reported NH-HAZE reference points are PSNR 17.081 for a CNN baseline and SSIM 0.602 for local airlight estimation, from the NH-HAZE benchmark paper. For ultra-high-resolution non-homogeneous haze, DehazeDCT reports 21.73 dB PSNR and 0.743 SSIM on the NTIRE 2024 data. We will run HazeFlow both as released and fine-tuned on temporally extended synthetic haze from its own Brownian-motion simulator, with matched budget.
Three ablations isolate the key ingredient. The first removes the transport constraint, leaving an independent density per frame. The second keeps transport but uses a fixed adjacency with no learned flow. The third replaces the dual network with fixed penalty weights tuned by grid search, which is the DC3-style alternative.
Beating the baselines means two things on the aligned static-camera benchmark described below. Our student must exceed fine-tuned DCL and fine-tuned HazeFlow in PSNR and SSIM, without losing LPIPS. This matters because Ensemble Dehazing Networks shows fidelity and perceptual winners can diverge. The no-transport ablation must also be measurably worse in both image fidelity and recovered relative depth. If it is not, the core claim fails.
Why not just…?
Why not just use the clearest frame from the same static camera as the label? This is the most serious alternative, because a fixed camera makes clear-day frames nearly aligned for free. It fails in three ways we can measure. Illumination and airlight differ between days. Vehicles and pedestrians differ between frames. Long periods of persistent haze may contain no clear frame. It is a real threat in the subset of scenes with stable lighting. We will therefore train a student on clear-frame pseudo-labels, with color transfer for illumination, under the same budget. We will report results separately for scenes binned by illumination difference and foreground density. If clear-frame supervision matches ours across bins, the physics stage is not worth its cost and we will say so.
Why not just run a strong single-image dehazer per frame and smooth over time? Single-image methods cannot separate density from depth, which is the ambiguity the field identifies as ill-posed without temporal geometry. Temporal smoothing also suppresses exactly the fog motion that carries information. HazeFlow plus temporal filtering is included as a baseline. The synthetic depth-ratio experiment will show whether per-frame methods recover relative depth at all.
Why not just extend the Brownian haze simulator over time and train fully supervised on synthetic clips? Synthetic transport with known truth removes the label problem but brings in a simulator-to-reality gap. HazeFlow itself needed a reflow stage on real hazy images to close that gap even for still images. We include a supervised synthetic-video baseline. If it matches our method on real footage, our contribution reduces to the simulator, and the experiment will show this.
Why not just add the physics as fixed-weight penalty terms? This is the cheaper version of our idea. PDL's comparison against DC3 on QP (0.176%–0.324% versus 4%–6% optimality gap) suggests that learned multipliers matter when constraints are tight. That evidence comes from optimization, however, not imaging. The fixed-penalty ablation, with its penalty weights grid-searched under the same budget, decides the question here. We will report transport-constraint violation alongside image quality.
Proposed Experiments
The decisive benchmark needs aligned ground truth on video, and the file contains none for static cameras. We will build it following the NH-HAZE protocol: a tripod, locked camera settings, haze machines, and fans. We extend that protocol in three ways. We record video while fans drive the haze across the scene for several minutes. We capture haze-free frames before and after. We acquire laser-measured depth for a subset of scenes. No result on this benchmark exists yet ("no reported result found on Lacuna" for any method).
The first decisive check is the transport ablation on this benchmark. We measure PSNR, SSIM and LPIPS of the student, together with scale-invariant relative depth error of the primal network. The second is the identifiability curve on synthetic clips, plotting depth-ratio error against mean fog displacement per frame. The physics predicts failure at zero motion and improvement as motion grows. If no such trend appears, the mechanism is not doing what we claim. The third compares warm-start against shrink-and-perturb for streaming windows on hour-long clips.
For generalization, the student is evaluated in single-frame mode on NH-HAZE, against the reported 17.081 PSNR and 0.602 SSIM reference points. It is also evaluated on unpaired real footage with FADE, NIQE and BRISQUE, using GoProHazy and DrivingHazy frames only as a stress test outside the static-camera regime, where we expect DCL to remain stronger. Finally, we measure whether dehazed camera features improve vehicle detection mAP on our foggy static-camera clips. This tests whether the recovered images are useful downstream, not only pleasant to look at.
References
- Driving-Video Dehazing with Non-Aligned Regularization for Safety Assistance
- Depth-Centric Dehazing and Depth-Estimation from Real-World Hazy Driving Video
- HazeFlow: Revisit Haze Physical Model as ODE and Non-Homogeneous Haze Generation for Real-World Dehazing
- Non-Homogeneous Haze Removal via Artificial Scene Prior and Bidimensional Graph Reasoning
- Self-Supervised Primal-Dual Learning for Constrained Optimization
- Learning for Dynamic Combinatorial Optimization without Training Data
- Continuous Learned Primal Dual
- GMAN: A Graph Multi-Attention Network for Traffic Prediction
- T-GCN: A Temporal Graph Convolutional Network for Traffic Prediction
- NH-HAZE: An Image Dehazing Benchmark with Non-Homogeneous Hazy and Haze-Free Images
- DehazeDCT: Towards Effective Non-Homogeneous Dehazing via Deformable Convolutional Transformer
- Ensemble Dehazing Networks for Non-homogeneous Haze