Super-Resolution Reconstruction of Remote Sensing Images of the China–Myanmar Pipeline Based on Generative Adversarial Network

Yuanliang Jiang, Qingying Ren, Yuan Ren, Haipeng Liu, Shaohua Dong, Yundong MaSustainability 2023 · DOI

TL;DR

This paper tackles the challenge of low-resolution remote sensing imagery for monitoring the China-Myanmar oil and gas pipeline. They introduce a novel Generative Adversarial Network (GAN) architecture, the Nonlocal Dense Receptive Field GAN (RRNL-SRGAN), which significantly enhances image detail and clarity, improving pipeline safety monitoring efficiency.

What Problem Does This Paper Address?

Ensuring the safety and operational integrity of critical infrastructure like the China-Myanmar oil and gas pipeline is paramount. The pipeline traverses challenging mountainous and vegetated terrain in Myanmar, making traditional ground-based inspections dangerous, inefficient, and time-consuming. Satellite remote sensing offers a broader scope and better timeliness for monitoring. However, the spatial resolution of available satellite imagery is often insufficient to discern crucial details for effective safety monitoring. This limitation hinders the ability to detect subtle changes, potential hazards, or anomalies along the pipeline route. Therefore, there's a clear need for advanced image processing techniques to enhance the resolution and detail of remote sensing data for this specific application.

Methods

The authors propose a novel Generative Adversarial Network (GAN) architecture, building upon the Super-Resolution Generative Adversarial Network (SRGAN) framework. Their approach introduces several key innovations:

  • Dense Residual Structure: To improve network depth and its ability to capture complex image features, they incorporate a dense residual structure.
  • Softsign Activation Function: To mitigate the common deep learning issues of easy saturation and gradient disappearance, they utilize the Softsign activation function.
  • Residual-in-Residual Nonlocal (RRNL) Dense Block: This novel module is designed to extract deep image features by combining residual-in-residual structures with a nonlocal mechanism. The nonlocal component is crucial for capturing long-range dependencies within the image, which is particularly relevant for remote sensing data.
  • Receptive Field Block (RFB) Mechanism: Integrated at the end of the generator, the RFB mechanism is employed to extract global image features, further enhancing the model's understanding of the scene.
  • Multi-Loss Function: To improve training stability and the quality of reconstructed images, the model utilizes a combination of four loss functions: adversarial loss, perceptual loss, mean square error loss, and total variance loss.

The discriminator is a standard convolutional network designed to distinguish between real high-resolution images and those generated by the network.

Datasets

The study utilizes a custom dataset of remote sensing images specifically collected from the Muse section of the China-Myanmar pipeline. These images were sourced from Google Earth and processed into 1100 images of size 512x512. The dataset was divided into training and validation sets with a 10:1 ratio. Low-resolution (LR) images were generated by performing a quadruple downsampling of the high-resolution (HR) images.

Key Insights and Ideas

  • Nonlocal Mechanisms are Key for Remote Sensing: The paper highlights the importance of nonlocal operations in capturing long-range spatial dependencies, which are inherent in remote sensing imagery of large geographical areas. This is a significant advantage over traditional convolutional approaches that focus on local neighborhoods.
  • Combining Global and Local Feature Extraction: The RRNL-SRGAN effectively combines the strengths of dense residual blocks for local feature extraction with the nonlocal mechanism for global context, leading to more comprehensive image understanding.
  • Targeted Architectural Improvements for SR: The specific modifications, such as the Softsign activation and the RRNL block, are tailored to address common challenges in super-resolution tasks, particularly for complex image types like remote sensing data.
  • Multi-Loss Strategy for Robust Training: The use of multiple loss functions demonstrates a sophisticated approach to guide the GAN training process, balancing fidelity, perceptual quality, and structural integrity.

Key Results

The proposed RRNL-SRGAN model achieved state-of-the-art performance on the custom China-Myanmar pipeline dataset:

  • Quantitative Metrics: The model achieved a Peak Signal-to-Noise Ratio (PSNR) of 30.20 dB and a Structural Similarity Index Measure (SSIM) of 0.840.
  • Comparative Improvements:
    • Compared to conventional methods (Bicubic, Nearest), the model showed significant improvements in PSNR (8.33 dB and 5.66 dB respectively) and SSIM (0.078 and 0.217 respectively).
  • Compared to SRCNN and SRGAN, the proposed method also demonstrated superior performance, with improvements of 1.41 dB and 0.21 dB in PSNR, and 0.059 and 0.006 in SSIM, respectively.
  • Qualitative Results: The reconstructed images exhibit significantly improved texture clarity, sharper edges, and more realistic details, making them more visually interpretable for human operators. This is particularly evident in the reconstruction of houses, roads, and natural features like terraces and water bodies.

Limitations and Future Work

  • Dataset Specificity: While the model performs well on the Muse section of the China-Myanmar pipeline, its generalization to other remote sensing datasets or different geographical terrains might require further validation.
  • Computational Cost: GANs, especially those with complex architectures like RRNL-SRGAN, can be computationally intensive for training.
  • Future Research Directions: The authors suggest continuing research to address detail loss in super-resolution and to explore land subsidence prediction to further enhance pipeline safety. This implies a broader scope for applying and extending their work to proactive hazard mitigation.

Summary updated

Abstract

The safety monitoring and early warning of overseas oil and gas pipelines are essential for China, as these pipelines serve as significant energy channels. The route of the China–Myanmar pipeline in Myanmar is mountainous and covered with vegetation, changeable climate, and abundant rain, and is prone to disasters. Therefore, artificial route inspection is dangerous and inefficient. Satellite remote sensing technology has an advantage over a traditional ground survey due to its large range and long-distance capabilities, thus can aid in monitoring the safety of oil and gas transportation. To improve the resolution of remote sensing data, in this paper, we propose a Nonlocal dense receptive field generative adversarial network, using remote sensing images of the Muse section of the China–Myanmar pipeline as data. Based on super-resolution generative adversarial network (SRGAN), we use a dense residual structure to improve the network depth and introduce the Softsign activation function to optimize the problem of easy saturation and gradient disappearance. To extract deep features of the image, we proposed a residual-in-residual nonlocal (RRNL) dense block, followed by the addition of the receptive field block (RFB) mechanism at the end to extract global features. Four loss functions are combined to improve the stability of model training and the quality of reconstructed image. The experimental results show that the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) of the reconstructed remote sensing image of Muse section reach 30.20 dB and 0.84. Compared to conventional methods and generic deep neural networks, the proposed approach achieved an improvement of 8.33 dB and 1.41 dB in terms of PSNR and an improvement of 21.7% and 5.9% in terms of SSIM. The reconstructed images exhibit improved texture clarity and are more visible to the human eye. This method successfully achieves super-resolution reconstruction of remote sensing images for the Muse section of the China–Myanmar pipeline, enhances the details of the image, and significantly improves the efficiency of satellite remote sensing monitoring.
BibTeX
@inproceedings{jiang2023super,
  author = {Yuanliang Jiang and Qingying Ren and Yuan Ren and Haipeng Liu and Shaohua Dong and Yundong Ma},
  title = {{Super-Resolution Reconstruction of Remote Sensing Images of the China–Myanmar Pipeline Based on Generative Adversarial Network}},
  booktitle = {Sustainability},
  year = {2023},
  doi = {10.3390/su151713068},
}
Figure