Abstract
Single Image Super-Resolution (SISR) aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs. While traditional GAN-based methods like SRGAN produce sharp textures, they often suffer from training instability and lower Peak Signal-to-Noise Ratio (PSNR). This paper proposes an improved SRGAN architecture utilizing Wasserstein GAN with Gradient Penalty (WGAN-GP) for stability and a hybrid loss function. By incorporating VGG16 for feature extraction and balancing pixel-wise Mean Square Error (MSE) with adversarial loss, the proposed model achieves both high mathematical fidelity and superior visual quality.
Introduction
Single Image Super-Resolution (SISR) is a fundamental challenge in computer vision. Traditionally, researchers faced a trade-off between mathematical accuracy and visual realism. MSE-based methods like SRCNN achieve high PSNR scores but often produce blurry images. Conversely, GAN-based methods like SRGAN generate sharper details but are difficult to train and often sacrifice PSNR. This research seeks to bridge this gap by redesigning the SRGAN framework to be more stable and effective.
Proposed Architecture and Methodology
The authors introduce several key modifications to the standard SRGAN framework to enhance performance and stability:
- Generator Network: The generator utilizes a residual convolutional neural network to facilitate faster convergence. It employs sub-pixel convolution (pixel shuffling) for upsampling to reconstruct the final HR image.
- Discriminator in Feature Space: Instead of operating on raw pixels, the discriminator evaluates high-level features extracted by a pre-trained VGG16 network. This allows the model to focus on structural content and edges rather than pixel-level noise.
- WGAN-GP Integration: To address the vanishing and exploding gradient problems common in GAN training, the authors implement Wasserstein GAN with Gradient Penalty (WGAN-GP). This approach constrains the discriminator (or "critic") to be 1-Lipschitz, leading to significantly more stable training.
- Hybrid Loss Function: The loss function is a weighted combination of pixel-wise MSE and adversarial loss. This balance ensures the reconstructed image remains faithful to the original structure while gaining realistic high-frequency textures.
Experimental Setup
The model was trained on the Pascal VOC 2012 dataset using 96x96 sub-images. Evaluation was conducted on three standard benchmarks: Set5, Set14, and BSD100. All experiments targeted a 4× up-sampling factor, which represents a significant challenge for detail recovery.
Key Results
The proposed model, SRWGAN-GP, consistently outperformed Bicubic interpolation, SRCNN, and the original SRGAN across all test sets in both PSNR and Structural Similarity Index (SSIM).
| Dataset | Metric | Bicubic | SRCNN | SRGAN | Ours (SRWGAN-GP) |
|---|---|---|---|---|---|
| Set5 | PSNR | 28.43 | 30.07 | 29.40 | 30.76 |
| SSIM | 0.8211 | 0.8627 | 0.8472 | 0.8791 | |
| Set14 | PSNR | 25.99 | 27.18 | 26.02 | 27.48 |
| BSD100 | PSNR | 25.94 | 26.68 | 25.16 | 26.81 |
Qualitatively, the model demonstrates a superior ability to recover fine structural details. For instance, in city scenes, the model successfully reconstructs the distinct patterns of windows on distant buildings that appear blurred in MSE-only models.
Conclusion
By combining the stability of WGAN-GP with a feature-space discriminator and a hybrid loss function, the authors successfully improved the state-of-the-art in SISR. The results indicate that the traditional trade-off between PSNR and visual sharpness can be mitigated through careful architectural design. Future work will explore deeper network structures and optimization techniques to further enhance detail recovery.
Summary updated