DGL-GAN: Discriminator Guided Learning for GAN Compression

Yuesong Tian, Li Shen, Dacheng Tao, Zhifeng Li, Wei Liu

Introduction

Generative Adversarial Networks (GANs) like StyleGAN2 and BigGAN have set the standard for high-resolution image synthesis. However, their massive parameter counts and high computational requirements (MACs) pose significant challenges for deployment on edge devices. DGL-GAN introduces a streamlined approach to GAN compression by leveraging a pretrained teacher discriminator to provide adversarial supervision, bypassing the pitfalls of traditional generator-to-generator distillation.

The Challenge of GAN Compression

Compressing noise-to-image GANs is difficult due to three primary factors:

  1. Sensitivity: Pruning generators often leads to immediate quality degradation.
  2. Lack of Ground Truth: Unlike image-to-image tasks, there is no single "correct" output for a given noise vector, making pixel-wise imitation ineffective.
  3. Training Instability: The adversarial nature of GANs makes adding complex teacher-student constraints prone to collapse.

Discriminator Guided Learning (DGL)

The core innovation of DGL-GAN is "Discriminator-to-Generator" guidance. Instead of forcing a student generator to match a teacher generator's features, the student is trained against both its own discriminator and a fixed, pretrained teacher discriminator. This teacher discriminator acts as a proxy for the real image distribution, providing a more stable and informative gradient signal.

The training follows a two-stage process:

  • Stage I: The narrow student GAN is trained from scratch.
  • Stage II: The student is fine-tuned with additional guidance from the teacher discriminator.

Key Results

DGL-GAN achieves state-of-the-art results across multiple benchmarks, including FFHQ, LSUN-church, and ImageNet.

FID vs. MACs comparison showing DGL-GAN (labeled DLG-GAN) outperforming baselines like Anycost and CA-GAN.

As shown in the figure above, DGL-GAN maintains lower FID scores (better quality) at significantly lower computational costs compared to existing methods. For StyleGAN2 on FFHQ, the model was compressed to 1/3 of its original parameters while maintaining an FID of 3.37, very close to the original 2.84.

Qualitative Quality

Beyond quantitative metrics, DGL-GAN preserves the semantic richness of the latent space. Even at 1/4 width, the model supports smooth latent space editing for attributes like age and gender while maintaining high visual fidelity.

High-quality face synthesis achieved by the compressed DGL-GAN model on the FFHQ dataset.

Conclusion

DGL-GAN demonstrates that the discriminator is a superior teacher for GAN compression. By focusing on distribution alignment rather than feature imitation, the method achieves significant speedups and parameter reductions without sacrificing image quality. Furthermore, applying this guidance to uncompressed models has even pushed StyleGAN2 to new state-of-the-art performance levels, achieving an FID of 2.65 on FFHQ.

Summary updated

Abstract

Generative Adversarial Networks (GANs) with high computation costs, e.g., BigGAN and StyleGAN2, have achieved remarkable results in synthesizing high resolution and diverse images with high fidelity from random noises. Reducing the computation cost of GANs while keeping generating photo-realistic images is an urgent and challenging field for their broad applications on computational resource-limited devices. In this work, we propose a novel yet simple {\bf D}iscriminator {\bf G}uided {\bf L}earning approach for compressing vanilla {\bf GAN}, dubbed {\bf DGL-GAN}. Motivated by the phenomenon that the teacher discriminator may contain some meaningful information, we transfer the knowledge merely from the teacher discriminator via the adversarial function. We show DGL-GAN is valid since empirically, learning from the teacher discriminator could facilitate the performance of student GANs, verified by extensive experimental findings. Furthermore, we propose a two-stage training strategy for training DGL-GAN, which can largely stabilize its training process and achieve superior performance when we apply DGL-GAN to compress the two most representative large-scale vanilla GANs, i.e., StyleGAN2 and BigGAN. Experiments show that DGL-GAN achieves state-of-the-art (SOTA) results on both StyleGAN2 (FID 2.92 on FFHQ with nearly $1/3$ parameters of StyleGAN2) and BigGAN (IS 93.29 and FID 9.92 on ImageNet with nearly $1/4$ parameters of BigGAN) and also outperforms several existing vanilla GAN compression techniques. Moreover, DGL-GAN is also effective in boosting the performance of original uncompressed GANs, original uncompressed StyleGAN2 boosted with DGL-GAN achieves FID 2.65 on FFHQ, which achieves a new state-of-the-art performance. Code and models are available at \url{https://github.com/yuesongtian/DGL-GAN}.
BibTeX
@article{tian2025dgl,
  author = {Yuesong Tian and Li Shen and Dacheng Tao and Zhifeng Li and Wei Liu},
  title = {{DGL-GAN: Discriminator Guided Learning for GAN Compression}},
  journal = {The Visual Computer},
  year = {2025},
  doi = {10.1007/s00371-024-03682-y},
}
Figure