Abstract
Layout-to-image (L2I) generation requires models to synthesize complex scenes from bounding boxes and labels, balancing global coherence with local detail. We introduce ObjBlur, a model-agnostic curriculum learning strategy that progressively reduces object-level blur during training. By starting with low-frequency structural information and gradually introducing high-frequency details, ObjBlur stabilizes the training process and mitigates augmentation leaking. Our experiments on COCO-Stuff and Visual Genome demonstrate that ObjBlur significantly improves image quality and recognizability across GAN and Diffusion-based architectures, achieving a 20% FID improvement on LostGAN and a 34.43% SceneFID improvement on LayoutDiffusion.
1. Introduction
Layout-to-image (L2I) generation is a challenging computer vision task where a model must generate a realistic image based on a provided layout of bounding boxes and semantic labels. This task requires the generator to simultaneously manage global scene composition and the fine-grained textures of individual objects. Current state-of-the-art models, including Generative Adversarial Networks (GANs) and Diffusion models, often struggle with training instability and "augmentation leaking," where training noise or transformations persist in the final generated output.
While curriculum learning (CL)—the practice of training models on increasingly difficult tasks—has been successfully applied to image resolution, its application to the semantic and spatial complexities of L2I generation has remained largely unexplored. We propose ObjBlur, a "plug-and-play" curriculum learning strategy that leverages object-level blurring to guide the model from coarse structural learning to fine-grained detail synthesis.
2. Methodology: The ObjBlur Strategy
ObjBlur is implemented within the data loading pipeline, making it compatible with any L2I architecture. The core innovation lies in applying blur selectively and progressively.
2.1 Object-Level Blurring
Unlike traditional methods that might blur an entire image, ObjBlur utilizes the layout masks provided in L2I datasets. It applies Gaussian blur specifically to foreground objects or the background, controlled by a probability parameter $p_{obj}$. This selective blurring forces the model to respect semantic boundaries and improves the separation between different elements in a scene.
2.2 The Curriculum Schedule
The "difficulty" of the generation task is modulated by the blur strength. At the beginning of training, images are heavily blurred, removing high-frequency details and leaving only low-frequency structural information. This encourages the model to first master the spatial arrangement of the layout.
As training progresses, the blur strength is reduced according to a schedule function $s(t)$. The authors evaluated several functions:
- Linear: Constant reduction in blur.
- Step: Periodic drops in blur intensity.
- Exponential: Rapid initial reduction.
- Sinusoidal: This schedule was found to be the most effective, providing a gentle "warm-up" phase and a smooth transition toward the final, clean images.
Crucially, because the blur strength reaches zero by the end of the training cycle, the model does not "learn" to produce blurry images, effectively solving the problem of augmentation leaking.
3. Experimental Setup
The effectiveness of ObjBlur was evaluated on two benchmark datasets:
- COCO-Stuff: Includes 80 "thing" classes and 91 "stuff" classes, providing a balanced mix of objects and backgrounds.
- Visual Genome: A highly complex dataset with dense layouts (averaging 3 to 30 objects per image) and intricate spatial relationships.
The strategy was integrated into several baseline models, including LostGAN, CAL2IM, and LayoutDiffusion, to demonstrate its model-agnostic nature.
4. Results and Analysis
The inclusion of ObjBlur led to significant quantitative and qualitative improvements across all tested architectures.
4.1 Quantitative Performance
- Fréchet Inception Distance (FID): On COCO-Stuff, ObjBlur improved the FID of LostGAN from 27.34 to 21.92, representing a ~20% improvement in overall image quality.
- SceneFID: To measure object-level quality, the researchers calculated FID on cropped objects. LayoutDiffusion saw a 34.43% relative improvement in SceneFID on the COCO dataset.
- Classification Accuracy Score (CAS): This metric measures how recognizable the generated objects are to a pre-trained classifier. On Visual Genome, LayoutDiffusion’s CAS increased from 36.45 to 40.15.
4.2 Training Stability
A key finding was the reduction in variance between training runs. GANs, which are notoriously sensitive to hyperparameters and initialization, showed much more consistent performance when trained with the ObjBlur curriculum.
5. Discussion and Conclusion
ObjBlur demonstrates that "structure before detail" is a powerful heuristic for L2I generation. By progressively introducing complexity, the model learns to prioritize layout coherence before attempting to synthesize realistic textures.
5.1 Limitations and Future Work
While effective, ObjBlur currently applies a uniform schedule to all objects. Future research could explore dynamic schedules that account for the inherent difficulty of specific classes (e.g., "human" vs. "sky"). Additionally, exploring other forms of degradation, such as additive noise or geometric distortions within a similar curriculum framework, could yield further improvements.
In conclusion, ObjBlur offers a simple yet highly effective method for improving the performance and stability of layout-to-image models, setting a new standard for curriculum learning in generative tasks.
Summary updated