Abstract
Generative Adversarial Networks (GANs) are powerful tools for image synthesis but suffer from training instability and uninformative loss metrics. This paper evaluates the application of Wasserstein GANs (WGAN) and WGAN with Gradient Penalty (WGAN-GP) to the task of face super-resolution. By reconstructing 64x64 high-resolution faces from 16x16 inputs using the CelebA dataset, we demonstrate that WGAN-GP provides superior training stability across architectures and a loss metric that correlates with perceptual quality, effectively mitigating common issues like mode collapse.
Introduction
Single Image Super-Resolution (SISR) aims to recover high-frequency details from low-resolution inputs. While standard GANs can produce sharp images, they are notoriously difficult to train due to three primary hurdles: training instability, uninformative loss values, and mode collapse. This study investigates whether Wasserstein-based objectives can overcome these challenges in the specific domain of facial reconstruction.
Methodology
The researchers compared three primary training objectives:
- Standard GAN: Utilizing Jensen-Shannon (JS) divergence, which often leads to vanishing gradients or oscillating training.
- WGAN (Weight Clipping): Approximating Wasserstein distance by enforcing a 1-Lipschitz constraint via weight clipping.
- WGAN-GP (Gradient Penalty): An improved WGAN variant that enforces the Lipschitz constraint by penalizing the gradient norm of the discriminator (critic).
To ensure the reconstructed faces remained faithful to the low-resolution inputs, an $L_1$ perceptual loss term was added to the objective. This term penalizes the pixel-wise difference between the downsampled generated image and the original input. Experiments were conducted on the CelebA dataset, downsampling 64x64 labels to 16x16 inputs.
Training Stability and Loss Interpretability
A core finding of this work is the reliability of the Wasserstein loss metric. In traditional GANs, the discriminator loss rarely reflects the actual quality of generated images. In contrast, the loss in WGAN-GP decreases monotonically as the model improves, providing a clear signal for convergence and hyperparameter tuning.
Furthermore, WGAN-GP demonstrated remarkable robustness to architectural changes. While standard GANs failed when using simple Multi-Layer Perceptrons (MLPs) or removing Batch Normalization, WGAN-GP consistently produced reasonable results, suggesting it significantly lowers the "black magic" required for GAN architecture design.
Experimental Results
Qualitatively, both WGAN and WGAN-GP outperformed traditional bicubic interpolation and standard GANs. The Wasserstein-based models produced sharper features, such as more defined eyes and realistic accessories like glasses, which were often blurred or distorted in standard GAN outputs.
Quantitatively, WGAN-GP effectively addressed mode collapse. When generating images from pure noise, WGAN-GP produced a diverse array of facial features and expressions, whereas standard GANs tended to collapse toward a limited set of "safe" variations.
Discussion and Future Work
Despite the improvements, several challenges remain. The models showed a bias toward frontal faces, likely due to the distribution of the CelebA dataset, performing less effectively on side profiles. Additionally, while the $L_1$ loss ensures input fidelity, it sometimes results in noisier images compared to purely adversarial objectives, highlighting a trade-off between pixel-wise accuracy and human visual preference. Future work could explore more diverse datasets and hybrid loss functions to further refine the balance between sharpness and structural integrity.
Summary updated