Cameras on underwater robots see a scene through water that acts as a color filter. In open ocean, red light dies within a few meters. In coastal water full of dissolved organic matter, the order flips and blue dies first. A fleet of robots spread across different sites therefore sees images that are corrupted in opposite ways. The robots also cannot easily pool their images: they talk over acoustic modems that carry less than 10 kbps, and survey data is often proprietary. Federated learning is the natural tool here. Each robot trains on its own images and sends only model updates to a server, which averages them. Naive averaging, however, would blend a "boost red" correction from ocean robots with a "boost blue" correction from coastal robots and serve neither.
Our proposal splits the restoration model into two parts. The first is a small shared network that every robot trains jointly and that handles structure, detail and the spatial shape of the degradation. The second is a private, per-robot description of its water, and that description is forced to lie on the physically possible set of color-attenuation values. Akkaynak and colleagues showed that this set is essentially a pair of lines in RGB space, one for oceanic water and one for coastal water (What is the Space of Attenuation Coefficients in Underwater Computer Vision?). Each robot's personalization therefore reduces to about five numbers that it fits locally. A new robot arriving at an unseen site can calibrate from a handful of unlabeled images without touching the shared weights.
Core Claim
We claim that constraining the private part to the measured attenuation manifold does three things that generic personalization cannot, while matching it on in-distribution clients. First, it lets a new client adapt from a few unlabeled images. Second, it extrapolates to water types that no training client saw. Third, it keeps the communicated model small enough for acoustic links.
The comparison that makes this claim testable is an unconstrained private code of the same size. If a free five-dimensional vector per client does as well as the physically bounded one, the physics adds nothing, and we will report that outcome.
What Exists and What Does Not
The only work in the file that federates learning across underwater vehicles is System-Level Optimization of AUV Swarm Control and Perception. It keeps sensor-specific encoders private and aggregates only a shared "mapper". This avoids what the authors call semantic mismatch, and it reaches 0.94 target-identification area under the curve against 0.86 and 0.88 for federated baselines. Their sensors are bathymetry and magnetics, not cameras, and their private parts are ordinary unconstrained networks.
We reuse their split-aggregation idea, which is the method-supplying role of that paper. We replace the private encoder with a physically parameterized one. Decentralized collaborative multi-institutional PET attenuation and scatter correction using federated deep learning federates attenuation correction for gamma-ray imaging. Its attenuation depends on tissue density rather than on wavelength, so it offers precedent for federated physics correction but no color model.
On the restoration side, every learned method in the file is trained centrally. This includes the 8.9K-parameter Five A+ Network (FA+Net) and the 0.28M-parameter FUSION. FUSION also hard-codes the assumption that red is the worst channel, giving it the smallest kernel. That assumption contradicts the coastal inversion documented by Akkaynak et al., and it is exactly the kind of fixed bias that fails when clients span water types; this tension motivates our design.
Domain Adaptation for Underwater Image Enhancement addresses the gap across water bodies, but it does so by pooling all images centrally. The benchmark paper An Underwater Image Enhancement Benchmark Dataset and Beyond documents that real references come from human voting. This constrains us: real-site clients cannot be assumed to hold clean targets.
Research Program
Each client models its images with a per-channel version of the standard haze model:
$$I_c(x) = J_c(x)\,t_c(x) + A_c\,(1 - t_c(x)), \qquad t_c(x) = e^{-\beta_c d(x)}$$
The terms are defined as follows:
- $I_c$ is the observed value in color channel $c \in \{R,G,B\}$.
- $J_c$ is the clean scene.
- $A_c$ is the ambient water color.
- $d(x)$ is the distance to the scene point at pixel $x$.
- $\beta_c$ is the effective attenuation in channel $c$.
The private constraint is $\beta = \beta_0^{(m)} + s\,u^{(m)}$. Here $m$ picks the oceanic or the coastal line, $\beta_0^{(m)}$ and $u^{(m)}$ are that line's anchor point and direction, and the scalar $s$ is the client's position along it.
The lines are computed once per camera model by projecting the public NASA SeaBASS spectra onto the camera's color sensitivities, as Akkaynak et al. did. No private data is needed for this step. The client's full private state $(m, s, A)$ is therefore one discrete choice plus four numbers.
The shared network is built by modifying FA+Net. It receives the image together with the private state, which modulates its features through per-channel scale-and-shift layers. It predicts a coarse distance map and a restored image. Because Akkaynak et al. show that attenuation also drifts with range and object color (by up to 14% for red between 5 and 10 m), the shared network may add a small bounded per-pixel correction to $\beta$. The manifold fixes the global color regime while the network handles the residual.
Training alternates. Clients first fit their private state on local data with the shared weights frozen, then take local gradient steps on the shared weights. Only the shared weights are averaged. The loss is supervised where paired references exist. Where they do not, a re-rendering loss is used: the restored image is pushed back through the physical model with the client's own $(\beta, A, d)$ and must reproduce the input. This lets unlabeled clients contribute.
A new client fits $(m, s, A)$ from a few images by minimizing that same re-rendering error. It never updates the shared network.
The communicated payload is roughly 35 KB at 32-bit precision, which is about 28 seconds per update at 10 kbps. We will also test 8-bit updates, following the quantization result reported by the swarm paper.
Sand-dust and tissue imaging have the same per-channel structure but no measured manifold of their own in the file. We treat them as a later extension, not part of the core test.
Baselines
The naive reference points are two FA+Net models. The first is trained purely locally on each client with no communication. The second is a pooled centralized model, which serves as an upper reference for what sharing could buy if privacy and bandwidth were ignored. For the centralized reference we cite the reported numbers: FUSION reaches 23.717 dB PSNR, 0.883 SSIM and 0.112 LPIPS on UIEB, and FA+Net reaches 23.061 dB PSNR on the T90 split. For any federated underwater enhancement method, no reported result was found on Lacuna, so all federated numbers will be produced by us under one protocol.
The first federated baselines are plain federated averaging applied to FA+Net and to FUSION. This is what a practitioner would try first.
The strongest competitor is generic split personalization. Clients keep FA+Net's 1×1 color-mapping layers private and share the rest, mirroring the mapper-centric design of the swarm paper, with a variant that uses the private enhancement blocks of pFLFE. This is the most dangerous rival because it also isolates color correction per client, but it does so without physics.
The second strong competitor is clustered federated learning, which groups clients by the similarity of their updates and trains one model per group, as surveyed in Clustered Federated Learning: A Review and Stochastic Clustered Federated Learning. It could discover the ocean/coastal split on its own.
The key ablation keeps our architecture and training schedule identical but replaces the manifold-constrained state with a free five-dimensional code. A second ablation drops the per-pixel correction to $\beta$.
All methods share these conditions:
- the same client partitions;
- the same number of communication rounds;
- the same local steps per round;
- a byte budget matched to FA+Net's size, with FUSION reported separately at its own size;
- the same hyperparameter search of equal trial count, tuned on held-out clients only.
Beating the generic personalization baseline means three things together. Our method must be within 0.2 dB PSNR of it on in-distribution clients. It must be at least 1 dB PSNR better on new clients given five or fewer images. It must show a lower color-error measure on held-out coastal clients. Each result must hold over three seeds.
Why not just…?
Why not just personalize the color layers generically and skip the physics? This is the real threat. The mapper-centric result in the swarm paper shows that private encoders already fix averaging conflicts. Our argument is narrower. A free private module has many degrees of freedom, so it needs labeled or plentiful local data to fit, and it has no reason to generalize to a water type absent from training. A point on a measured line has one continuous degree of freedom inside each regime. The free-code ablation and the few-shot and new-water-type tests settle this directly. If generic personalization matches us there, we drop the physical-constraint claim and report the split design alone.
Why not just let clustered federated learning find the water types? Clustering by update similarity can recover two coarse groups. It still averages within a group, though, and water inside the "coastal" set spans a wide range along its locus. It also gives a new client no way to join without first computing updates on labeled data. We will run clustering with an oracle given the true Jerlov type, which is its best case, and compare within-cluster spread of error against our continuous position $s$.
Why not just white-balance or boost channels with fixed per-site multipliers? Akkaynak et al. show that effective attenuation shifts with distance and object color, so a single multiplier per site leaves range-dependent hue errors. The UIEB study reports that prior-based methods trail learned ones on full-reference metrics. We include a per-client gray-world plus fitted fixed-gain baseline so that this gap is measured, not assumed.
Why not just recover the vehicles and train centrally afterward? When data can legally be pooled and adaptation can wait, this is the better option, and our centralized reference quantifies its advantage. The proposal targets the other case: cross-organization surveys and long missions where the swarm paper reports 4.5× more information per joule than streaming.
Proposed Experiments
We run two client settings.
In the first, controlled setting, NYU-v2 depth scenes are degraded synthetically, as UIEC^2-Net did. Each of 20 clients is assigned attenuation values sampled along the oceanic or the coastal locus with varied ambient color, which gives exact references. Two whole regions of the coastal locus are held out as unseen water types.
In the second, real setting, each dataset acts as a site-client: UIEB, EUVP, LSUI, SUIM-E and UFO-120. Clients are split further by degradation where labels exist. U45's green, blue and hazy categories and SQUID's real dive sites serve as new clients, the latter unpaired. Paired sets are scored with PSNR, SSIM and LPIPS, and all sets with UCIQE and UIQM. We also log bytes per round and simulated acoustic time.
The decisive check is new-client adaptation from 1, 5 and 20 unlabeled images. We compare manifold-fitted state, free-code fitting and fine-tuning the generic private layers, measured on the held-out coastal regions of the synthetic setting and on SQUID and U45.
A second check tests the plain-averaging failure itself. We measure how far the hue of FedAvg outputs drifts relative to local-only training as the fraction of coastal clients grows from zero to one half. If plain averaging shows no such drift, the premise of the proposal is weaker than the literature suggests, and we will say so.
A third check verifies that the fitted $(m, s)$ recovers the true water regime on synthetic clients. This confirms that the private state is being interpreted physically rather than acting as an arbitrary code.
References
- What is the Space of Attenuation Coefficients in Underwater Computer Vision?
- System-Level Optimization of AUV Swarm Control and Perception: An Energy-Aware Federated Meta-Transfer Learning Framework with Digital Twin Validation
- Decentralized collaborative multi-institutional PET attenuation and scatter correction using federated deep learning
- Five A+ Network: You Only Need 9K Parameters for Underwater Image Enhancement
- FUSION: Frequency-guided Underwater Spatial Image recOnstructioN
- Domain Adaptation for Underwater Image Enhancement
- An Underwater Image Enhancement Benchmark Dataset and Beyond
- pFLFE: Cross-silo Personalized Federated Learning via Feature Enhancement on Medical Image Segmentation
- Clustered Federated Learning: A Review
- Stochastic Clustered Federated Learning