Hyperspectral images are data cubes with dozens of narrow wavelength channels per pixel, and they are often sensitive: a cube can reveal the chemical makeup of vegetation, the mineral composition of a site, or tissue properties in medical imaging. Because these cubes are expensive to capture and annotate, the natural training regime is federated — institutions each hold a few private scenes and share only averaged parameter gradients with a central server. The security of that arrangement rests on a belief that gradient inversion attacks, which reconstruct training data from shared gradients, have already been broken at scale: modern attacks recover images from batches of 100 or more on classification benchmarks. But every scalable attack in the literature leans on machinery that hyperspectral models do not have. They exploit cross-entropy gradients at a classification head, where labels are discrete and the head's weight gradients decompose into interpretable outer products (Inverting Gradients - How easy is it to break privacy in federated learning?, Instance-wise Batch Label Restoration via Gradients in Federated Learning, SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning). Hyperspectral super-resolution (HSSR) models are dense regressors: they map RGB or coarse spectral images to full 31-band cubes under $L_1$ or spectral losses, with no softmax, no class labels, and volumetric 3D convolutions or spatial-spectral attention in place of wide fully connected layers (Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution, SSHFormer: Optimizing Spectral Reconstruction with a Spatial–Spectral Hybrid Transformer).
This proposal asks whether such models leak their private training cubes, and answers with a two-sided hypothesis. On the attack side, the very physics that makes hyperspectral data compressible — every pixel spectrum is approximately a non-negative mixture of a few pure material signatures — is a prior the attacker can exploit, shrinking the search space from $H \times W \times 31$ free pixels to a low-rank factorization and enforcing physically plausible spectra that generic image priors cannot. On the science side, the literature contains a direct contradiction that this setting can resolve: one line argues that higher-dimensional models leak more because gradients become overdetermined (Inverting Gradients - How easy is it to break privacy in federated learning?), while the hyperspectral literature warns that high-dimensional volumetric optimization collapses into local minima (SSHFormer: Optimizing Spectral Reconstruction with a Spatial–Spectral Hybrid Transformer). Nobody knows which force wins when the data is a 31-channel cube and the model has 3D separable kernels. The closest existing work inverts multimodal RGB-plus-text models (MMGIA: Gradient Inversion Attack Against Multimodal Federated Learning via Intermodal Correlation, Gradient Inversion of Multimodal Models) or trains compressive spectral imagers cooperatively without any privacy analysis (Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging); none touches dense spectral regression, and none uses physical mixing constraints as an inversion prior.
Core Claim
The claim is that gradient inversion scales to hyperspectral dense-regression models once the attacker optimizes in the data's natural low-rank subspace rather than in pixel space. Concretely, instead of searching for a free cube $\hat{X} \in \mathbb{R}^{H \times W \times C}$ that matches the shared batch gradient, the attacker searches over an endmember–abundance factorization
$$\hat{X} \approx \hat{E}\hat{A}, \qquad \hat{E} \ge 0, \quad \hat{A} \ge 0, \quad \textstyle\sum_p \hat{A}_{p,j} = 1,$$
where $\hat{E} \in \mathbb{R}^{C \times P}$ holds $P$ candidate material spectra (endmembers) and each column of $\hat{A}$ gives a pixel's mixing proportions, with $P \ll C$. This parameterization comes straight from the linear mixing model that underpins coupled spectral unmixing for super-resolution (Hyperspectral Super-Resolution by Coupled Spectral Unmixing) and from the low-rank tensor view of hyperspectral data (Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization); the proposal turns a modeling assumption of one field into an attack regularizer for the other. Two further ingredients complete the method. First, since there is no classification head, the attack matches gradients by direction (cosine similarity, following Inverting Gradients - How easy is it to break privacy in federated learning?) but replaces the spatial total variation prior with a spectral one: a smoothness penalty along the wavelength axis plus a spectral-angle term that keeps recovered curves within physically sensible shapes. Second, the linear output layer of a regression network offers an analytical foothold that partially replaces the missing label-restoration step: with an $L_2$ loss and a linear head, the weight gradient aggregates as $\nabla_W = \sum_i \mathbf{h}_i (\hat{y}_i - y_i)^\top$, an outer-product sum over hidden features and per-sample residual vectors — structurally the same object that SPEAR: Exact Gradient Inversion of Batches in Federated Learning and SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning disaggregate into individual samples on fully connected classification layers, except that the role of the one-hot label vector is played by the continuous residual. Whether that analogy survives contact with 3D separable convolutions and attention is exactly what the research program tests.
Research Program
The program has three components. The first builds the attack itself, which we call unmixing-regularized gradient inversion: parameterize each dummy batch sample by the factorization above, initialize endmembers by a crude spectral clustering of any public hyperspectral data (the endmember dictionary does not need to match the victim's scenes, only to span a plausible spectral subspace), and optimize the cosine-similarity objective over $(\hat{E}, \hat{A})$ with per-batch-sample abundances, using spectral total variation across wavelength rather than only across space. The number of endmembers $P$ is a single hyperparameter, swept against the victim model's implicit rank. The second component works out the analytical layer: we derive what information the output-layer gradient sum exposes about the aggregated residuals and hidden features in MCNet-style and SSHFormer-style architectures, and attempt a SPEAR++-style dictionary disaggregation there, with ReLU replaced by whatever activation the victim actually uses. The honest expectation, grounded in the stated limitation of SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning that its formulation is restricted to fully connected layers with piecewise-linear activations, is that the pure analytical route will fail on these architectures; the deliverable is a characterization of why and a hybrid that uses the analytical step only to warm-start the subspace optimization. The third component is the scaling study that resolves the contradiction in the literature: sweep batch size $B$ from 1 to 256, number of spectral bands $C$ from 3 (RGB control) to 31, and victim capacity, and measure where recovery quality saturates or collapses. This yields the first empirical curve of privacy risk versus data dimensionality for dense regression, which is the scientifically novel output independent of whether the attack fully succeeds.
Baselines
The naive reference point is recovering the batch mean or a dataset-mean cube, which sets the PSNR floor any real attack must clear, together with Euclidean-distance gradient matching of the original DLG style: on trained networks this was reported at roughly 7 dB PSNR versus about 14 dB for cosine matching on CIFAR ResNet-20 in Inverting Gradients - How easy is it to break privacy in federated learning?, so it should fail badly here too. The strongest existing competitor is the cosine-similarity attack of Inverting Gradients - How easy is it to break privacy in federated learning? run unchanged on the hyperspectral victim models: it is architecture-agnostic, needs no classification head, and has already reconstructed images at batch size 100 on trained ResNet-152, so it is the right representative of the current state of the art. We will run it on identical victim models, identical shared gradients, identical data splits, and give it the same GPU-hours, iteration budget, and learning-rate tuning sweeps as our method, with its spatial total variation weight tuned as carefully as our spectral priors. The comparison metric is deliberately two-sided: an attack that wins on PSNR but produces spectrally incoherent cubes has not recovered a hyperspectral image, so beating the baseline means beating it on Spectral Angle Mapper (SAM, which measures the angular error between recovered and true per-pixel spectra) while matching or beating its PSNR at each batch size, and remaining stable at batch sizes where the baseline collapses. A second strong baseline is SPEAR++ (SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning), which achieved exact pixel-perfect recovery up to $B = 210$ on CIFAR-10 but only on fully connected ReLU layers; running it against the separable-3D-convolution and attention victims quantifies the architectural wall that motivates our optimization-first design. Finally, the key ablation isolates the physical prior: our full attack versus the identical pipeline with the endmember–abundance parameterization removed and the cube optimized freely in pixel space with a 3D total variation penalty of matched strength. If the factorization is doing real work, the ablated version should degrade specifically in SAM and at large $B$; if it does not, the proposal's central claim is wrong and we would report that.
Why not just…?
Why not just run standard 2D gradient inversion band-by-band? The band-by-band idea treats each of the 31 channels as an independent grayscale image and applies an off-the-shelf 2D attack. Two things break. Empirically, the hyperspectral literature shows that ignoring inter-band correlation destroys spectral fidelity: Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution attributes 2D-only models' spectral distortion to exactly this, and Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization documents that 2D spatial total variation cannot model cross-channel continuity. Structurally, the bands are not even separable at the gradient level: in a 3D separable convolution the spectral filters mix channels before the spatial ones, so the shared gradient of a spectral filter entangles all bands in one update, and band-by-band inversion would have to fabricate the coupling. We will still run this baseline, and predict it recovers spatially plausible but spectrally meaningless cubes — high PSNR, poor SAM — which is precisely the failure our unmixing prior is designed to fix.
Why not just add differential privacy noise or prune the gradients? As a defense question this is legitimate, and we treat it as part of the study rather than an objection to dodge. But the file's evidence says naive deployments do not stop scalable attacks: Instance-wise Batch Label Restoration via Gradients in Federated Learning reports that differential privacy and gradient sparsification had surprisingly little impact on analytical label recovery (above 96% accuracy retained), SPEAR: Exact Gradient Inversion of Batches in Federated Learning maintains exact reconstruction under gradient clipping and DP noise that defeats other attacks, and SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning confirms robustness to DP noise. There is one honest caveat in the other direction: noise calibrated for high-dimensional hyperspectral gradients may behave differently than on CIFAR-scale models, and our defense experiments (DP-SGD and pruning sweeps at several noise levels against our strongest attack configuration) are designed to measure the privacy–utility trade-off rather than assume it.
Why not just rely on large batches, since hyperspectral training already aggregates many patches? This is directly refuted by the gradient-inversion literature at every scale: Catastrophic Data Leakage in Vertical Federated Learning reconstructs images at 31.83 dB PSNR from batch size 40 on CIFAR-10 (versus 9.29 dB for the older DLG approach) and reports leakage up to $B = 100$; Inverting Gradients - How easy is it to break privacy in federated learning? recovers identifiable images at $B = 100$; SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning achieves exact recovery at $B = 210$; and Instance-wise Batch Label Restoration via Gradients in Federated Learning extracts batch composition at $B = 4096$. The open question is not whether large batches are safe in classification, but whether the regression setting shifts the threshold, and our batch sweep answers that empirically rather than by analogy.
Proposed Experiments
All experiments place a client-side HSSR model in a federated loop and give the attacker only the averaged mini-batch gradient, the architecture, and current weights. Victim models are MCNet, whose mixed 2D/3D convolutional design and reported CAVE results (39.026 dB PSNR, 0.9319 SSIM, 3.292 SAM at $\times 4$ scale in Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution) make it the standard deep baseline, and SSHFormer, the spatial-spectral hybrid transformer that reached 34.39 dB PSNR on the NTIRE 2022 dataset in SSHFormer: Optimizing Spectral Reconstruction with a Spatial–Spectral Hybrid Transformer; a small fully connected ReLU victim is included as a positive control where SPEAR++'s exact machinery should apply, with its reported pixel-perfect recovery (PSNR above 100 dB up to $B = 25$, and $B = 210$ in the SPEAR++ extension) as the ceiling reference. Data are CAVE (32 indoor scenes, 31 bands, 400–700 nm), Harvard (77 scenes, 31 bands), the NTIRE 2022 spectral recovery pairs, and one remote-sing scene from the HYDICE set used by Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization to test cross-domain endmember dictionaries. Metrics are PSNR and structural similarity for spatial fidelity, SAM for spectral fidelity (the primary privacy metric, since a leaked cube with wrong spectra leaks no material signature), and attacker wall-clock time, benchmarked against SPEAR's reported 2 minutes for a batch of 20 versus over 18 minutes for optimization baselines at roughly 19.6 dB (SPEAR: Exact Gradient Inversion of Batches in Federated Learning). No reported result exists on Lacuna for gradient inversion on any hyperspectral benchmark, so all numbers in this study will be new. Three checks are decisive. First, the prior-isolation ablation must show that removing the endmember–abundance parameterization degrades SAM and large-batch stability while leaving small-batch PSNR roughly intact; otherwise the physical prior is decoration. Second, the dimensionality sweep must produce a monotone or saturating relationship between spectral band count and recovery quality under our attack, resolving the overdetermination-versus-optimization-collapse contradiction with data; a collapse at moderate $C$ would falsify the core claim and itself be a publishable negative result with clear guidance for practitioners. Third, the defense sweep must locate the noise level at which DP-SGD genuinely blocks spectral recovery, giving hyperspectral federated deployments the first quantified privacy budget for their setting rather than an imported CIFAR-calibrated rule of thumb.
References
- Inverting Gradients - How easy is it to break privacy in federated learning?
- Instance-wise Batch Label Restoration via Gradients in Federated Learning
- SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning
- Mixed 2D/3D Convolutional Network for Hyperspectral Image Super-Resolution
- SSHFormer: Optimizing Spectral Reconstruction with a Spatial–Spectral Hybrid Transformer
- MMGIA: Gradient Inversion Attack Against Multimodal Federated Learning via Intermodal Correlation
- Gradient Inversion of Multimodal Models
- Cooperative Hardware-Prompt Learning for Snapshot Compressive Imaging
- Hyperspectral Super-Resolution by Coupled Spectral Unmixing
- Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization
- SPEAR: Exact Gradient Inversion of Batches in Federated Learning
- Catastrophic Data Leakage in Vertical Federated Learning