Abstract
The discriminator plays a vital role in training generative adversarial networks (GANs) by distinguishing real and synthesized samples. While the real data distribution remains constant, the synthesis distribution varies as the generator evolves, fundamentally changing the classification task for the discriminator. We propose DynamicD, a training strategy that adjusts the discriminator's capacity on-the-fly to accommodate this time-varying task. Our empirical study confirms that DynamicD improves synthesis performance without additional computational costs. We develop two capacity-adjusting schemes: progressively increasing capacity for sufficient data regimes and gradually decreasing layer width for limited data regimes to mitigate overfitting. DynamicD is synergistic with existing discriminator-improving approaches and demonstrates substantial gains across 2D and 3D-aware image synthesis tasks.
The Challenge: A Moving Target
In standard GAN frameworks, the discriminator maintains a fixed architecture. However, the difficulty of its task—telling real images from fake ones—is non-stationary. Early in training, the generator produces crude noise, making the discriminator's job trivial. As the generator matures, the "fake" samples become increasingly sophisticated, requiring a more powerful discriminator to provide useful gradients.
This static approach leads to two primary failure modes:
- Under-fitting (Sufficient Data): When training data is plentiful, a fixed-capacity discriminator may eventually lack the representational power to keep pace with a highly evolved generator, stalling progress.
- Over-fitting (Limited Data): In low-data regimes, a large discriminator often "memorizes" the small training set rather than learning generalizable features. This leads to a collapse in synthesis quality as the discriminator becomes too proficient at identifying specific training samples.
Methodology: DynamicD
To address these issues, DynamicD adjusts the discriminator's width (number of filters) during training. This alignment between model complexity and task difficulty ensures stable competition throughout the GAN process.
1. Increasing Capacity (Sufficient Data)
For large datasets, the discriminator starts with a reduced width (e.g., 50% of the baseline). As training progresses, new filters are initialized and added linearly until the model reaches full capacity. This "growth" phase mirrors the increasing complexity of the generator's output.
2. Decreasing Capacity (Limited Data)
To combat memorization in data-scarce environments, the discriminator starts at full capacity and gradually shrinks its width. This is achieved by randomly dropping a subset of filters at each step, acting as a form of weight-level dropout. This prevents the model from latching onto specific training samples and encourages the learning of more robust features.
Crucially, DynamicD incurs zero additional computational cost because the model is always smaller than or equal to the baseline size during training.
Experimental Results
The researchers evaluated DynamicD across several benchmarks, including FFHQ (faces), LSUN (scenes), and Carla (3D-aware synthesis).
- Low-Data Regimes: On the FFHQ 0.1k dataset, DynamicD achieved a massive improvement, reducing the Fréchet Inception Distance (FID) from 179.21 (StyleGAN2 baseline) to 50.37.
- High-Data Regimes: Even on massive datasets like LSUN Bedroom, where baselines are already strong, DynamicD pushed the FID from 5.65 down to 4.01.
- 3D-Aware Synthesis: Applied to StyleNeRF on the Carla dataset, it improved FID from 53.87 to 47.42, demonstrating that the strategy generalizes beyond 2D pixels.
Conclusion
DynamicD offers a simple yet powerful paradigm shift: matching the discriminator's capacity to the evolving difficulty of the GAN training task. By treating the discriminator as a dynamic component rather than a static one, we can significantly improve image quality and training stability across diverse data regimes and architectures. Future work may explore automated scheduling via AutoML or extending these principles to depth-based or Transformer-based architectures.
Summary updated