TL;DR
SPS-GS (Sub-Pixel-Shifted Gaussian Splatting) is a new framework for arbitrary-scale remote sensing super-resolution that achieves multi-frame performance with the efficiency of a single-pass encoder. By synthesizing sub-pixel shifted frames during training and employing 2D Gaussian Splatting for continuous rendering, it effectively recovers high-frequency details from aliased signals while remaining significantly faster than traditional implicit neural representation (INR) methods.
What Problem Does This Paper Address?
Remote sensing imagery is often constrained by sensor resolution, yet downstream applications—like environmental monitoring or geographic analysis—frequently require images at various spatial scales. This is the domain of Arbitrary-Scale Super-Resolution (ASSR).
Current approaches fall into two camps, both with significant drawbacks:
- Single-Image ASSR: These methods use a single low-resolution (LR) observation. Because LR sampling discards high-frequency information, the problem is underconstrained, and the results often lack fine detail.
- Multi-Frame SR: These methods use multiple observations of the same scene to exploit complementary sub-pixel information. However, they typically require repeated feature encoding for every frame and specialized multi-frame training sequences, leading to high computational costs and data acquisition hurdles.
Furthermore, most multi-frame methods treat aliasing (the jagged artifacts caused by under-sampling) as noise to be suppressed. The authors of this paper argue that aliasing actually contains valuable high-frequency information if you can correctly demodulate the phase shifts between frames.
Methods
The authors propose SPS-GS, an "encode-once" framework that integrates multi-frame information before the deep encoding pass.
The architecture consists of three main components:
- Sub-Pixel-Shifted Training: To avoid the need for real-world multi-frame training sets, the model synthesizes $K$ sub-pixel-shifted LR frames from a single high-resolution (HR) image during training. This allows the model to learn multi-frame fusion using standard single-image datasets.
- Phase-Coherent Aliasing Demodulation (PCAD): This module aligns non-reference frames to a reference grid using Fourier-domain phase modulation. It then uses a phase-aware branch and regularized least-squares estimation to extract and fuse the complementary aliasing information. Crucially, this happens before the main encoder, allowing the network to "encode once."
- Gaussian Projection Alignment (GPA): Instead of using standard pixels or MLPs, the model represents the image as a set of 2D Gaussians. GPA iteratively refines the centers and colors of these Gaussians by back-projecting reconstruction errors from all $K$ observations. This ensures the final HR output is consistent with all input LR frames.
Datasets
The model was evaluated on five standard remote sensing benchmarks: AID, UCMerced, WHU-RS19, RSSCN7, and NWPU-RESISC45.
To test real-world performance, the authors also evaluated the model on three multi-frame datasets:
- PROBA-V: Satellite imagery at $\times 3$ scale.
- WorldStrat: Multi-temporal satellite imagery at $\times 4$ scale.
- RealBSR-RGB: A burst super-resolution dataset.
Key Insights and Ideas
- Aliasing as a Signal, Not Noise: By recognizing that sub-pixel shifts introduce distinct phase modulations to aliased components, the authors turned a sampling artifact into a source of high-frequency data.
- Efficiency through Splatting: Traditional ASSR methods use Implicit Neural Representations (INRs), which require querying a heavy MLP for every single pixel in the output image. By using 2D Gaussian Splatting, the model can render high-resolution images much faster, as splatting is a highly efficient rasterization process.
- Synthetic-to-Real Transfer: The paper demonstrates that a model trained entirely on synthetic sub-pixel shifts can generalize effectively to real-world multi-frame satellite data, significantly lowering the barrier for training multi-frame SR models.
Key Results
SPS-GS consistently outperformed both single-image and multi-frame baselines across all datasets and scales.
- Accuracy: On the AID dataset at $\times 4$ scale, SPS-GS achieved a PSNR of 32.72 dB, outperforming the best multi-frame baseline (ST-AVSR) by 0.45 dB and single-image methods by over 3 dB.
- Generalization: The model showed strong performance at "unseen" scales (e.g., $\times 8$ and $\times 16$) that were not encountered during training.
- Speed: Due to the "encode-once" design and Gaussian rendering, SPS-GS is roughly 7x to 10x faster than INR-based decoders like LIIF or LTE at common scales.
Limitations and Future Work
While SPS-GS is a significant step forward, the authors note a few areas for improvement:
- Radiometric Consistency: The current model assumes frames have consistent brightness and contrast. Real satellite imagery often suffers from radiometric variations between passes.
- Environmental Factors: The framework does not yet explicitly handle cloud interference or significant parallax (3D) effects caused by different viewing angles in satellite bursts.
- Refinement Iterations: While GPA improves accuracy, the iterative nature of the refinement adds some computational overhead, though it remains faster than many competitors.
Summary updated