Abstract
Text-to-image synthesis has seen significant progress with Generative Adversarial Networks (GANs). However, many models suffer from "discursive generation," where objects and backgrounds blend inconsistently. We present DGattGAN, a Dual Generator attentional GAN that decouples object and background synthesis. By employing a cooperative up-sampling mechanism and asymmetric information feeding, DGattGAN generates high-quality images with sharp outlines and high text consistency. Experiments on the CUB and Oxford-102 datasets demonstrate that DGattGAN outperforms state-of-the-art methods in both Inception Score (IS) and R-precision.
Introduction
Generating realistic images from natural language descriptions is a complex task requiring both semantic understanding and high-fidelity visual synthesis. A persistent challenge in GAN-based approaches is the lack of clear separation between the target object and its environment. This often leads to blurry boundaries and "ghosting" effects. DGattGAN addresses this by splitting the generation process into two specialized paths, ensuring that each component of the scene receives appropriate focus.
Methodology
Dual Generator Architecture
DGattGAN utilizes two distinct generators: $G_O$ for the primary object and $G_B$ for the background. This separation allows the model to focus on fine-grained object details without being distracted by background complexity.
Cooperative Up-sampling
A major risk in dual-generator systems is "degeneration," where one generator (typically the background one) becomes obsolete if the object mask expands to cover the entire frame. DGattGAN introduces a cooperative up-sampling mechanism where feature maps are shared between $G_O$ and $G_B$ via $1 \times 1$ convolutions before each up-sampling step. This ensures mutual information flow and prevents either generator from collapsing.
Asymmetric Information Feeding
To maintain the functional distinction between generators, DGattGAN employs asymmetric information feeding. The object generator ($G_O$) is provided with full word-level and sentence-level features to capture intricate details. Conversely, the background generator ($G_B$) receives "starved" information through a dropout layer, preventing it from attempting to synthesize object-specific features in the background regions.
Experimental Evaluation
The proposed model was evaluated on the Caltech-UCSD Birds (CUB) and Oxford-102 Flowers datasets.
Quantitative Results
DGattGAN achieved an Inception Score (IS) of 4.45 on the CUB dataset, surpassing AttnGAN (4.36). More significantly, it reached an R-precision of 62.45%, representing a substantial improvement in text-image alignment over previous state-of-the-art models.
Qualitative Analysis
Visual results demonstrate the effectiveness of the decoupling strategy. Generated images show distinct boundaries and high fidelity to descriptive prompts.
As illustrated above, the model excels at translating complex textual attributes into precise visual features. The attention mechanism on the object generator ensures that words like "short squat body" or "brownish white belly" are accurately localized.
Conclusion
DGattGAN introduces a robust framework for text-to-image synthesis by effectively decoupling object and background generation. Through cooperative up-sampling and asymmetric information feeding, the model overcomes the common pitfall of generator degeneration. While challenges remain regarding dataset-driven morphological biases, DGattGAN sets a new standard for realistic and text-consistent image generation.
Summary updated