Sharing the Weather: Decentralized Degradation Consensus for Camera Networks

A camera that restores its own image in bad weather first has to work out what is wrong with the picture: how dense the haze is, how heavy the rain is, whether there are drops on the lens. Today's all-in-one restoration models make that guess alone, from one corrupted frame, and the guess is weakest exactly when the frame is worst. Cameras in a network, such as the roadside units at an intersection, all look through the same weather. Each one also carries its own local problems: a wet or dirty lens, sensor noise, compression artifacts. We propose that neighboring cameras agree on the shared part of the degradation by exchanging a few hundred bytes with their peers. No pixels are sent and no central server is involved. Each camera then conditions a lightweight local restorer on the agreed-upon estimate.

The core technical piece is a robust consensus problem. Every node splits its local degradation estimate into a component common to the scene, which it agrees on with its neighbors, and a sparse private component that absorbs its own lens and sensor faults. We solve this with the same decentralized alternating-direction machinery power grids use to coordinate distributed generators without a master controller. The bet is simple and falsifiable: pooling evidence across views should fix the misdiagnoses that make single-image blind restoration fail. It should also hold up when nodes are blinded, when they drop out, and when the server that a centralized design would depend on goes down.

Core Claim

Compound degradations have a sharing structure across a camera network that has not been used. Multi-Agent Image Restoration (MAIR) organizes real-world corruption into three physical levels: scene-level (weather, haze, lighting), imaging-level (blur, sensor noise) and compression-level. It shows that respecting this order reduces error propagation. We observe that the scene level is largely shared among cameras viewing the same place at the same time, while the imaging and compression levels are specific to each camera. So a blind degradation estimate, which today is computed per image, can be split into a shared part and a private part. The shared part can be estimated collectively.

Our claim is that a decentralized, outlier-robust consensus on the shared part gives lower degradation-estimation error than any single node's estimate, and this in turn improves both restoration and downstream detection. The gain should be largest for the nodes whose own view is most corrupted. Because the exchanged object is a short degradation code rather than an image, the scheme fits within the payload sizes that CoInfra measured as reliable over commercial 5G: 1–2 KB payloads had 24.2 ms mean latency and 0% loss. Raw images at 500–600 KB had over 2.19 s latency and a 63.75% drop rate.

We should be upfront about which part of the gain comes from where. At this payload size, bandwidth no longer forces decentralization, since a central server could average codes just as cheaply. The accuracy gain comes from pooling. Decentralization contributes resilience: there is no single point of failure, and nodes can join or leave. We will measure these two contributions separately rather than blend them.

Mechanism

Each node $i$ runs a degradation encoder that outputs a code $c_i$. This is a vector that summarizes the corruption in its current frame. Nodes $i$ and $j$ are neighbors on a communication graph if their views overlap in time and weather. Each node keeps a shared-weather estimate $z_i$ and a private residual $e_i$, and the network jointly solves

$$\min_{\{z_i, e_i\}} \sum_i \Big( \|c_i - z_i - e_i\|^2 + \lambda \|e_i\|_1 \Big) + \mu \sum_{(i,j)} \|z_i - z_j\|^2 .$$

The first term asks the shared and private parts to explain the local code. The $\ell_1$ penalty with weight $\lambda$ keeps the private part sparse, so a lens-drop signature is absorbed into $e_i$ instead of contaminating the shared weather. The smoothness term with weight $\mu$ lets the shared estimate vary slowly across the graph instead of forcing exact equality. This matters because haze depth or sun direction can differ across a large intersection.

We solve the problem with decentralized ADMM (alternating direction method of multipliers, an optimizer that splits a global problem into local subproblems coordinated through exchanged multipliers). The design follows Decentralized Dynamic Power Management with Local Information. That work decomposes grid optimization into regional subproblems that exchange only boundary dual variables, uses network coloring to avoid message collisions, and converges to $10^{-3}$ relative error in about 40 iterations on a 69-bus feeder. Our variables are low-dimensional and the weather changes slowly compared with the frame rate, so consensus can be warm-started from the previous frame and run for a few iterations per update.

The restorer at each node is conditioned on $z_i + e_i$ in place of its own raw code. The architecture follows the explicit degradation-code modulation in AirNet, which learns a degradation embedding contrastively and injects it through deformable convolutions and feature transforms. We chose it over PromptIR because an explicit code is what we need to exchange. PromptIR's prompts are generated inside the network and are not separable into scene and camera factors. To make the split meaningful, we supervise the scene-level coordinates of the code on synthetic data with known weather parameters, such as haze scattering coefficient and rain or snow density. We keep AirNet's contrastive loss for the remaining coordinates.

Research Program

The first stage builds a multi-view compound-degradation benchmark where ground truth exists. We take clear-weather frames from CoInfra's 14 synchronized roadside nodes. We render shared weather onto all views of a timestep: haze uses a common scattering coefficient with transmission computed from the co-registered LiDAR depth, and rain or snow uses shared intensity. Onto each camera independently we add private degradations: adherent drops on a random subset of lenses, noise at node-specific ISO, and JPEG at node-specific quality. Rendering haze from true depth matters. Deep Joint Rain Detection and Removal from a Single Image shows that heavy rain is a non-additive mix of streaks and depth-dependent veiling. A benchmark that composites weather without depth would reward the wrong model.

The second stage trains the code-conditioned restorer and encoder on this benchmark and on the All-Weather corpus (Snow100K, Raindrop, Outdoor-Rain). This way the single-node restorer is competitive with published models before any consensus is added.

The third stage adds decentralized consensus. It studies three things: convergence speed, sensitivity to $\lambda$ and $\mu$, and behavior under node dropout and message loss. Consensus methods are known to have trouble with lossy, delayed links, as discussed in Fast frequency response with heterogeneous communication delay management under the SCION Internet architecture and Stable current sharing and voltage balancing in DC microgrids. Stage three therefore injects CoInfra-like latency and loss into the message channel rather than assuming ideal links.

The fourth stage moves to real CoInfra rain, snow and freezing-rain footage. No clean ground truth exists there, so we evaluate through camera-based 3D detection on the provided annotations.

We deliberately leave out learned penalty tuning, although A reinforcement learning approach to parameter selection for distributed optimal power flow reports a 59.3% iteration reduction for it on the IEEE 9-bus system. Our consensus problem is small and warm-started, so iteration count is not expected to be the bottleneck. We will add learned tuning only if stage three shows it is.

Baselines

The naive reference point is running camera detection on unrestored frames. This tells us whether restoration of any kind helps downstream perception in real weather, which is not guaranteed.

The strongest single-node competitor is a per-node blind all-in-one restorer. We use AirNet with its own local code, and PromptIR as a stronger architecture. Both are trained on exactly our synthetic multi-view training set and All-Weather, with the same number of training iterations and the same hyperparameter search budget as our model. They are the direct target: the only thing our method changes is where the code comes from.

For published reference numbers on All-Weather, M2Restore reports an average of 31.65 dB PSNR and 0.936 SSIM, at 0.17 s per 224×224 image on an RTX 3090. WeatherDiff needs about 20 s per 640×432 frame, and no reported PSNR for it was found on Lacuna. Our single-node restorer should come within about 1 dB of M2Restore on All-Weather while being cheaper. If it does not, the consensus gains would be measured on a weak base and would not be credible.

The pooling-without-decentralization baseline is a central server that collects all codes and returns their plain mean and their robust split. This separates the accuracy gain from the resilience gain.

The key ablation removes the private term: we set $\lambda \to \infty$, so the private residual is always zero and consensus reduces to smoothing raw codes. This isolates the claim that separating shared and private degradation is what prevents a blinded lens from poisoning its neighbors.

An oracle that conditions each restorer on the true rendered degradation parameters gives the upper bound. Beating the per-node baseline means two things on the synthetic benchmark: lower code error, and higher PSNR, SSIM and LPIPS on the worst-quartile nodes. On real CoInfra it means higher camera-detection [email protected] in snow and freezing rain. The only CoInfra number in the file is for LiDAR plus camera fusion: global multimodal VRU detection drops from 0.654 in sunny weather to 0.585 in heavy snow. No camera-only AP was found on Lacuna, so we establish that baseline ourselves.

Why not just…?

Why not just average the degradation code over time at a single camera? This is the most serious threat. Weather changes slowly, so temporal pooling at one node might remove most of the estimation noise with no communication at all. It cannot fix a persistent bias, though. A lens covered by a drop for minutes produces consistently wrong codes, and averaging them over time only averages the error. We will run temporal pooling with the same window length as our consensus and report the comparison split by whether a node's lens is contaminated. If temporal pooling matches us on contaminated nodes too, the cross-camera claim is false and we will say so. We also evaluate a combined space-time variant, because the honest outcome may be that the two are complementary.

Why not just send detections, as CoInfra already does? CoInfra's central fusion of 1–2 KB detections is reliable. But detection quality is bounded by what each camera's detector sees, and that is exactly what weather degrades. Exchanging degradation codes costs about as much bandwidth as detections and acts earlier in the pipeline. The two approaches also compose: consensus-restored images can feed the same detection fusion. Our real-weather experiment is designed as exactly this comparison, detection fusion with and without code consensus upstream.

Why not just use a stronger per-image restorer, or MAIR's central scheduler, and skip coordination? MAIR's own design shows that diagnosis drives the whole chain. Its scheduler's perception step decides which experts run, and it takes tens of seconds per image. Is Nano Banana Pro a Low-Level Vision All-Rounder? finds that large generative models trail specialists by 4–8 dB PSNR, which is a poor trade in safety-critical scenes. A stronger per-image model still diagnoses from one corrupted view. We test this directly: if PromptIR's advantage over AirNet disappears when both are given the oracle code, then diagnosis rather than capacity is the bottleneck, and fixing the code through consensus is the right lever.

Why not do true multi-view pixel fusion? Pixel fusion across wide-baseline roadside cameras requires dense cross-view warping and heavy transfers, and CoInfra shows those transfers collapse on 5G. We restrict sharing to the degradation, which is genuinely shared across views. The scene content, by contrast, is seen from different angles and is not simply shared.

Decisive Checks

The first check comes before any restorer is trained. On the synthetic benchmark, we plot degradation-code error against the number of neighbors and against the fraction of contaminated lenses, for local, temporal, plain-average, robust-central and robust-decentralized estimates. The proposal survives only if robust consensus beats local and temporal estimates on contaminated nodes, and decentralized matches central within noise.

The second check is whether lower code error turns into restoration gains. It succeeds if our method closes a meaningful share of the gap between the per-node baseline and the oracle.

The third check is on real CoInfra weather, measuring camera-detection AP with CoInfra-like message loss injected. We also switch off random nodes, and switch off the would-be central server, and track how gracefully accuracy degrades for the remaining nodes. A clean result here would show that the degradation estimation step of blind restoration can be moved from each image to the network, cheaply and without a central controller.