Abstract: Recently, satellites with high temporal resolution have fostered wide attention in various practical applications. Due to limitations of bandwidth and hardware cost, however, the spatial resolution of such satellites is considerably low, largely limiting their potentials in scenarios that require spatially explicit information. To improve image resolution, numerous approaches based on training low-high resolution pairs have been proposed to address the super-resolution (SR) task. Despite their success, however, low/high spatial resolution pairs are usually difficult to obtain in satellites with a high temporal resolution, making such approaches in SR impractical to use. In this paper, we proposed a new unsupervised learning framework, called "MIP", which achieves SR tasks without low/high resolution image pairs. First, random noise maps are fed into a designed generative adversarial network (GAN) for reconstruction. Then, the proposed method converts the reference image to latent space as the migration image prior. Finally, we update the input noise via an implicit method, and further transfer the texture and structured information from the reference image. Extensive experimental results on the Draper dataset show that MIP achieves significant improvements over state-of-the-art methods both quantitatively and qualitatively.
The Spatial-Temporal Trade-off in Satellite Imaging
In satellite imaging, there is a persistent trade-off: you can have high temporal resolution (frequent revisits, like video satellites) or high spatial resolution (fine details), but rarely both. High-frequency satellites often produce low-resolution images due to hardware and bandwidth constraints.
Standard Super-Resolution (SR) solutions typically require thousands of perfectly aligned pairs of low-resolution (LR) and high-resolution (HR) images to "learn" how to reconstruct details. In remote sensing, these pairs are incredibly difficult to acquire. While we might have an HR photo of a city from a previous year, our current high-frequency satellite provides only a blurry view of that city today. This paper introduces Migration Image Prior (MIP), an unsupervised framework that performs SR without needing matched training pairs.
Methods: Migrating Priors through Latent Space
The MIP framework treats a high-resolution reference image (taken at a different time) as a "prior"—a set of stylistic and structural rules—rather than a direct template. The process involves three core components:
- Generative Network: The model uses a generative adversarial network (GAN) structure that starts with random noise and iteratively learns to reconstruct the scene.
- Spatial Transformer Networks (STN): Because the reference image and the target image may have different shooting angles, STNs are integrated to learn geometric transformations, aligning the reference features with the target scene.
- Migration Image Prior: The model converts the reference image into a latent space and uses an implicit update method to modify the input noise maps. This effectively "migrates" sharp textures and structural information from the HR reference into the generation process.
To ensure accuracy, the model downsamples its high-resolution output and compares it to the original low-resolution input. If the downsampled version matches the original blurry input, the model confirms it is correctly reconstructing the underlying scene while adding migrated details.
Experimental Results
The study utilized the Draper dataset, focusing on scenarios in Southern California. The researchers used images from different days to test the model's ability to transfer information across time. MIP was compared against state-of-the-art unsupervised methods, including Deep Image Prior (DIP) and Zero-Shot Super-Resolution (ZSSR).
- Quantitative Performance: On a 4x upsampling task, MIP achieved a PSNR of 30.55 dB and an SSIM of 0.8388, outperforming the second-best method (DIP) by nearly 1 dB in PSNR and significantly improving structural similarity.
- Qualitative Clarity: Visually, MIP results show much sharper edges and finer details, such as road markings and building boundaries, compared to other unsupervised methods which often produce blurry artifacts or noise.
Discussion and Limitations
While MIP represents a significant leap in unsupervised SR, it relies on the availability of a "similar" high-resolution reference image. If the landscape has changed drastically—such as new construction or deforestation—the migration of old priors might introduce inaccuracies. Additionally, like many GAN-based iterative optimization methods, the process is computationally intensive, requiring thousands of iterations per image. Future work will focus on refining latent space representations to achieve more stable and efficient texture transfers.
Summary updated