Introduction
Text-to-image generation using Generative Adversarial Networks (GANs) has made significant strides, yet two major hurdles persist: the "diversity gap" and the "category consistency issue." The diversity gap refers to the tendency of models to produce nearly identical images for a single prompt, a phenomenon known as mode collapse. Category consistency involves the model's failure to adhere to the specific visual traits of a real-world category. For instance, a model might generate a realistic bird that lacks the specific color patterns defining its species, focusing on local text alignment while ignoring global visual concepts.
Methodology
The Category-Consistent and Relativistic Diverse Conditional GAN (CRD-CGAN) addresses these issues through a multi-faceted architecture consisting of three primary pillars.
Multi-Generator Framework (D-CGAN)
To combat mode collapse, CRD-CGAN utilizes $K$ generators sharing a single discriminator. Each generator receives a unique noise vector. A diversity loss function is applied, which maximizes the ratio between the distance in the generated image space and the distance in the latent noise space, forcing the generators to produce distinct visual outputs for the same text prompt simultaneously.
Relativistic Conditional Loss
Traditional GAN discriminators output an absolute probability of an image being real or fake. CRD-CGAN adopts a relativistic approach, estimating the probability that a real image is more realistic than the average synthetic image, and vice versa. This relative comparison provides more stable gradients during training and encourages the model to capture finer, more realistic details.
Category-Consistent Loss
The core innovation of CRD-CGAN is the category-consistency loss. By extracting features from both real and synthetic images and calculating their cosine similarity, the model ensures that synthetic images remain anchored to their ground-truth categories. This acts as a visual "anchor," preventing the generation of "generic" objects and ensuring that species-specific traits—such as the exact shade of black for an American Crow—are preserved.
Experimental Results
The model was evaluated on the CUB (Birds-200-2011), Oxford-102 Flowers, and MS COCO 2014 datasets, demonstrating superior performance across quality and diversity metrics.
Quantitative Performance
CRD-CGAN showed significant improvements in both diversity and quality:
- Diversity (LPIPS): On the CUB dataset, it achieved an LPIPS score of 39.91%, notably higher than AttnGAN (35.26%) and StackGAN++ (31.37%).
- Quality (FID): The model reached a competitive FID of 24.59 on CUB and a state-of-the-art 40.75 on the Oxford Flower dataset.
- Human Evaluation: In user studies, CRD-CGAN images were preferred for their realism significantly more often than baseline models (e.g., 33.24% preference vs. 12.38% for AttnGAN on birds).
Qualitative Insights
Visual results demonstrate that CRD-CGAN is highly sensitive to semantic nuances in text prompts. For example, changing a description from "white belly" to "yellow belly" results in accurate color shifts while the overall category-specific structure of the bird remains intact. This effectively solves the "Grey Crow" problem where previous models would often ignore category-specific colors in favor of generic bird features.
Conclusion and Future Work
CRD-CGAN effectively bridges the gap between text-to-image alignment and category-specific realism. While highly successful on fine-grained datasets like birds and flowers, complex multi-object scenes in datasets like MS COCO remain a challenge for GAN-based methods. Future research could extend category-consistency constraints to individual objects within multi-object scenes and optimize the computational efficiency of the multi-generator architecture for real-time deployment.
Summary updated