Super-resolution on remote sensing images

Yuting Yang, Kin‐Man Lam, Junyu Dong, Xin Sun, Muwei Jian2021 · DOI

Abstract: High-resolution ocean remote sensing images are critical for oceanographic research, yet available data often suffers from low resolution due to temporal averaging or cloud interference. This paper presents a modified Very Deep Super-Resolution (VDSR) model specifically optimized for satellite-derived Sea Surface Temperature (SST) images. By integrating features from multiple network depths, the proposed architecture enhances the reconstruction of complex ocean structures like coastal fronts. Evaluated on several AVHRR-based databases, the model demonstrates superior performance in both Peak Signal-to-Noise Ratio (PSNR) and perceptual loss compared to state-of-the-art models and traditional interpolation methods.

Introduction

In oceanography, high-resolution imagery is essential for studying air-sea interactions, weather prediction, and marine ecosystems. However, researchers often face a "resolution gap." Satellite data is frequently provided as weekly or monthly averages, which smooths out fine-grained details. Furthermore, "instant" satellite images are often plagued by cloud contamination, necessitating the use of lower-resolution composite products.

This research focuses specifically on ocean fronts—the boundaries between water masses with different physical properties like temperature and salinity. These fronts are vital for biodiversity, yet they are difficult to study when the resolution is low. While general-purpose super-resolution models exist for natural photos, they often struggle with the unique, turbulent textures of ocean temperature data, particularly in complex coastal regions.

Methodology

The authors build upon the Very Deep Super-Resolution (VDSR) model, a 20-layer convolutional neural network (CNN) that uses residual learning to predict the difference between a blurry low-resolution image and its high-resolution counterpart.

The "Modified" part of the approach involves integrating feature maps from multiple layers of the network. While deep layers capture abstract features, low-level layers contain the most reliable raw information from the input. By bridging these different levels of abstraction, the model can better reconstruct the intricate, non-linear patterns found in coastal waters.

The modified VDSR architecture integrates features from multiple layers to better capture non-linear patterns in ocean data.

To evaluate the results, two primary metrics were used:

  1. PSNR (Peak Signal-to-Noise Ratio): Measures pixel-to-pixel accuracy.
  2. Perceptual Loss: Uses a pre-trained VGG19 network to ensure the spatial structure and content of the reconstructed image are perceptually accurate, rather than just matching raw pixel values.

Datasets

The study utilized several specialized datasets derived from Advanced Very High Resolution Radiometers (AVHRR):

  • China Ocean SST Database: 600 daily averaged images covering China’s coastal waters.
  • Ocean SST Database: 1,500 images covering a broader range of the Pacific.
  • SST Ocean-Front Database: 1,500 images specifically processed to highlight ocean fronts using the Microcanonical Multiscale Formalism (MMF) method.
  • Set291: A standard collection of natural images used as a baseline for non-satellite data performance.

Key Results

Quantitative improvements were most dramatic in challenging coastal water tests. The modified VDSR achieved a PSNR of 21.53 dB and a remarkably low Perceptual Loss of 5.89, significantly outperforming the standard VDSR (17.67 dB PSNR, 53.77 Perceptual Loss) and traditional bicubic interpolation (17.04 dB PSNR, 21.07 Perceptual Loss).

Qualitatively, visual comparisons demonstrate that the modified model successfully eliminates the "blur" around coastal edges and complex fronts. This provides a much more accurate map for oceanographers to locate and study dynamic marine structures.

Qualitative comparison of SST reconstruction methods. The modified model (e) significantly reduces blur and artifacts in complex coastal regions compared to bicubic interpolation (b) and standard models.

Discussion and Future Work

A key insight from this study is that domain specificity matters. Interestingly, the authors found that the EDSR (Enhanced Deep Super-Resolution) model—which typically outperforms VDSR on natural images—did not perform as well on SST data. This suggests that "bigger is not always better" when dealing with specialized geophysical data, and architecture must be tuned to the specific textures of the domain.

Future work will focus on training with real-world pairs of low-resolution and high-resolution satellite images rather than simulated downsampled data. Additionally, the team plans to apply this architecture to other remote sensing data types, such as chlorophyll concentration and sea surface height.

Summary updated

Abstract

High-resolution ocean remote sensing images are of vital importance in the research field of ocean remote sensing. However, the available ocean remote sensing images are composed of averaged data, whose resolution is lower than the instant remote sensing images. In this paper, we propose a very deep super-resolution learning model for remote-sensing image super-resolution. In our research, we target satellite-derived sea surface temperature (SST) images, a typical kind of ocean remote sensing image, as a specific case study of super-resolution on remote sensing images. In this paper, we propose a novel model architecture based on the very deep super-resolution (VDSR) model, to further enhance its performance. Furthermore, we evaluate the peak signal-to-noise ratio (PSNR) and perceptual loss of the model trained on the natural images and SST frames. We designed and applied our model to the China Ocean SST database, the Ocean SST database, and the Ocean-Front databases, all containing remote sensing images captured by advanced very high resolution radiometers (AVHRR). Experimental results show that our model performs better than the state-of-the-art models on SST frames.
BibTeX
@misc{yang2021super,
  author = {Yuting Yang and Kin‐Man Lam and Junyu Dong and Xin Sun and Muwei Jian},
  title = {{Super-resolution on remote sensing images}},
  year = {2021},
  doi = {10.1117/12.2590197},
}
Figure