Spanner-Built Attention Graphs for Hyperspectral Fusion

Hyperspectral cameras record hundreds of narrow color bands, but each band receives few photons, so the pixels have to be large and the images come out blurry. Ordinary color or multispectral cameras give sharp images with only a few bands. Fusion methods combine a blurry many-band image with a sharp few-band image of the same scene to reconstruct a cube that is both sharp and spectrally rich. The best of them use non-local attention: each pixel borrows information from distant pixels made of the same material. Full attention costs grows with the square of the pixel count. Cheaper schemes pick each pixel's partners by nearest-neighbor search or clustering, and both tend to cut the image into islands of similar material. Pixels belonging to a small or rare material end up talking only to each other.

We propose building this sparse "who talks to whom" graph with a rule taken from wireless network design. In localized topology control, each radio picks its links using only nearby information so that the resulting network has bounded degree, stays connected, and keeps short paths between any two nodes. The rule we borrow is the Yao rule: split the space around each node into angular cones and keep only the nearest neighbor in each cone. We apply it to pixels placed in a low-dimensional space made of spectral-subspace coordinates plus spatial position. This gives every pixel the same small number of partners, as kNN does, but the partners point in different directions of material space. The graph therefore stays connected and information can cross between materials in a few layers. The claim is narrow and testable: at an equal edge budget, a cone-based graph should reduce spectral distortion more than kNN or clustering graphs, and the gain should be largest on small objects, material boundaries, and photon-starved (noisy) inputs.

Core Claim and Why It Is Different

The direct pairing of these two fields is absent from the literature we found. The closest work sits on either side.

SSDCN replaces quadratic attention in hyperspectral super-resolution with spatial and spectral K-means clustering. That cuts cost to $O(KN)$ and gives large memory savings on Pavia Center. Its clusters, however, are hard, global, and sensitive to the chosen number of clusters, and nothing guarantees that information flows between clusters. SSDCN is the method we must beat on efficiency-matched accuracy.

DynAGS learns per-node edge pruning with hard-concrete masks for traffic forecasting. It shows that adaptive sparse graphs beat static ones at matched sparsity, but it was applied to time series, not imaging cubes. Neither paper uses a selection rule with structural guarantees.

Topology control provides exactly such rules. Three results supply the method:

  • The 3D Symmetric Yao Graph bounds degree using only 1-hop coordinate information.
  • Generalized Gabriel constructions keep an edge only when a region between its endpoints is empty, which yields near-optimal power stretch.
  • Percolation analysis shows that global connectivity can survive a constant number of links per node, provided the local selection rule is well designed rather than "nearest k."

That last point is the crux of our argument against kNN.

The spectral side tells us where to put the cones. Scenes are governed by a few endmembers (pure materials) and their mixing proportions, as exploited by non-local sparse tensor factorization and coupled CPD with identifiability guarantees. Projecting pixels onto this low-rank subspace gives coordinates of dimension roughly 3 to 6. In that range cone partitions stay affordable, whereas cones in raw 100-band feature space would be hopeless.

This proposal differs from prior work in three ways. It uses a geometric spanner rule rather than clustering (SSDCN) or learned masks (DynAGS). It works in material space rather than pixel-grid space; on a pixel grid, spanners are trivial because the grid is already one. And it makes a falsifiable prediction about where the gains should appear.

Research Program

The graph is rebuilt at each non-local block, in three steps.

  1. Subspace coordinates. We take the low-resolution hyperspectral input, compute its leading $d$ spectral components by SVD, and project the current high-resolution feature map onto them. This gives each pixel $i$ a coordinate $z_i \in \mathbb{R}^d$. We then append scaled spatial position, $p_i = (z_i, \alpha x_i, \alpha y_i)$, where $\alpha$ trades material similarity against spatial proximity.
  2. Directional nearest neighbors. We fix $m$ unit direction vectors $u_1,\dots,u_m$ spread over the sphere. Each pixel assigns every candidate $j$ to the cone of the direction most aligned with $p_j - p_i$, and keeps the nearest candidate in each nonempty cone. Degree is therefore at most $m$, and attention costs $O(mN)$.
  3. Local candidate pools. To keep construction localized in the topology-control sense, each pixel searches only a candidate pool. The pool is its spatial window plus a fixed set of pixels from a coarse anchor grid. This mirrors a 2-hop neighborhood horizon, in the spirit of planarity from 2-hop information.

Graph selection is not differentiated, just as in kNN attention. Attention weights over the chosen edges are learned as usual.

We will plug this block into two backbones so the effect is not tied to one architecture. The first is a physics-unfolded network in the style of MHF-net, which turns iterative gradient steps on the camera degradation model into network stages; we insert our block inside its learned denoising step. The second is a plain residual transformer backbone.

Alongside reconstruction metrics we will log graph diagnostics:

  • the number of connected components;
  • the algebraic connectivity $\lambda_2$ of the graph Laplacian, which UAV consensus work ties to how fast information mixes;
  • hop distance between pixels of different materials.

These let us test the mechanism, not just the score.

The photon-budget direction enters as an evaluation axis rather than decoration. We will simulate fewer photons per band by adding signal-dependent noise to the low-resolution hyperspectral input at several levels. We expect non-local aggregation to matter more as noise rises. We also expect kNN in spectral coordinates to latch onto neighbors that share noise rather than material, while cone diversity should resist this.

Baselines

The naive reference is bicubic upsampling of the hyperspectral cube. The next step up is the backbone with only local convolutions. MHF-net reports roughly 32.25 dB for a generic ResNet on CAVE, which bounds what purely local processing achieves.

The strongest reported accuracy baseline in our file is MHF-net itself, at 37.23 dB PSNR on CAVE. LSO uses learned latent tokens and a sine/cosine basis to limit spectral distortion. It reports a 0.6% PSNR gain and a 6.9% reduction in spectral angle error at ×8 over the next best competitor, but the file gives no absolute dB value. We will retrain LSO with released settings where available rather than quote it.

For efficiency, SSDCN reports 0.51 s inference, 0.029 GB peak memory, and 1.31M parameters on Pavia Center at ×4. No PSNR for SSDCN appears on Lacuna, so we will reproduce it.

The decisive comparisons are graph-rule ablations inside one fixed backbone. All rules share the same training data, epochs, learning-rate search budget, and exactly matched edges per pixel. The rules are:

  • kNN in the same subspace coordinates;
  • kNN plus random long-range edges;
  • SSDCN-style cluster-centroid attention with matched cost;
  • shifted-window attention;
  • full attention on small crops, as an accuracy ceiling;
  • our cone rule with $m=1$, which collapses to 1-nearest-neighbor and isolates the role of directional diversity.

Beating the baselines means two things. First, at least a consistent 0.3 dB PSNR gain and a clear reduction in spectral angle error over the best matched-degree alternative, across three or more seeds and two backbones. Second, memory within a small factor of SSDCN's.

Why not just…?

Why not just use kNN attention in feature space? This is the main threat, because kNN also gives bounded degree. The percolation result in low-degree connectivity suggests that nearest-k rules fragment where a better-designed rule does not, but that evidence comes from radio networks, not images. Our component counts and $\lambda_2$ logs will show whether kNN graphs actually fragment on CAVE and Chikusei. The region-stratified errors will show whether fragmentation costs accuracy. If kNN neither fragments nor loses accuracy, the idea fails.

Why not just add random long-range edges to kNN? Small-world shortcuts do raise algebraic connectivity, as the UAV swarm study shows for consensus. This is a cheap and plausible fix, and it could match our method. The difference we predict is that random edges connect pixels arbitrarily, whereas cone edges connect each pixel to the nearest pixel in each material direction. Those edges are more useful for borrowing spectra. The matched-degree kNN-plus-random arm settles this directly. If it ties our method, we will report that the benefit is connectivity alone and not the spanner geometry.

Why not just use SSDCN's clustering? Clustering already achieves linear cost with good efficiency. However, its hard assignments give no route between clusters, and its quality depends on the hand-chosen cluster count. We will sweep SSDCN's cluster counts within the same tuning budget we give our cone count $m$, so neither method wins by tuning effort.

Why not just windowed attention or convolutions? SSDCN reports that window methods lose non-local context and blur edges. The ResNet-versus-MHF-net gap on CAVE points the same way. We include window attention anyway as the honest local reference.

Proposed Experiments

Datasets. We will train and test on CAVE and Harvard (31 bands, indoor and outdoor) and on the remote-sensing scenes Pavia Center, Chikusei, Botswana, and Washington DC Mall. Degradation is synthetic at ×4 and ×8 with known blur. A misregistration stress test shifts the sharp image by subpixel offsets, addressing the simulation gap noted by MHF-net. WorldView-2 is used only for qualitative real-sensor checks.

Metrics. We report PSNR, spectral angle mapper, and ERGAS, together with peak memory and latency.

Decisive checks. There are three.

  • Stratified regions. We stratify pixels using ground-truth endmember clusters into large-material interiors, small or rare materials, and boundaries. The cone graph should win mainly in the latter two strata and tie in large interiors. Uniform gains would suggest a generic regularization effect rather than our mechanism.
  • Noise sweep. Across the photon-starvation sweep, the gap between our method and kNN should widen as noise grows.
  • Mechanism check. Across scenes, our accuracy advantage over kNN should correlate with how fragmented the kNN graph is. If there is no such correlation, our explanation is wrong even if the scores improve.