Abstract
Text-to-image (T2I) diffusion models frequently struggle with semantic binding, where attributes and spatial relationships are incorrectly assigned to entities. We present R-Bind, a training-free inference-time optimization method that enhances both attribute and relation binding. By optimizing cross-attention maps during the denoising process using a set of targeted losses, R-Bind ensures that attributes align with correct entities and that spatial relationships are preserved. Our method achieves a 12.8% average improvement over state-of-the-art baselines across standard benchmarks and demonstrates compatibility with both U-Net and DiT-based architectures.
The Semantic Binding Challenge
Despite the impressive capabilities of models like Stable Diffusion, they often fail to accurately interpret complex prompts involving multiple objects and specific attributes. This "semantic binding" failure typically manifests in two ways:
- Entity-Attribute Binding: Misassigning colors or textures (e.g., swapping "green" and "blue" in "a green bench and a blue bowl").
- Entity-Relation-Entity Binding: Failing to correctly position objects relative to one another (e.g., "a man on the left of a lamp").
While previous work has addressed attribute binding, relation binding—the spatial and logical interaction between objects—remains a significant hurdle. Existing solutions often require costly model fine-tuning or lack generalizability across different model architectures.
Methodology: Inference-Time Optimization
R-Bind introduces a training-free strategy that guides the image generation process by manipulating attention maps. The framework employs three primary loss functions:
- Focus Loss ($L_{focus}$): Prevents object omission or merging by ensuring distinct regions of the image attend to specific prompt entities.
- Entity-Attribute Binding Loss ($L_{ea}$): Aligns attribute attention maps (e.g., "green") with their respective entity maps (e.g., "bench") while minimizing overlap with unrelated entities.
- Entity-Relation-Entity Binding Loss ($L_{ere}$): A novel loss that ensures relation tokens (e.g., "chasing") attend to both involved entities while maintaining the distinctness of those entities to prevent "hybrid" object generation.
The system utilizes a Large Language Model (Gemma-3-27B) to parse prompts into structured entities and relations. Optimization occurs during the first 50% of the denoising steps, nudging the latent representation toward a semantically accurate layout.
Experimental Evaluation
R-Bind was evaluated on T2ICompBench and GenAIBench, focusing on color and spatial accuracy.
Key findings include:
- Performance Gains: On Stable Diffusion 1.5, R-Bind nearly doubled the spatial relation accuracy on T2ICompBench, increasing the score from 8.7 to 15.6. It outperformed the strongest baseline, CONFORM, by a significant margin.
- Architecture Agnostic: The method is effective on both U-Net (SD-1.5) and Diffusion Transformer (SD-3) architectures. On SD-3, R-Bind improved the T2ICompBench Color score from 80.3 to 82.5.
- Human Preference: In a 100-prompt human evaluation, R-Bind was preferred over the base SD-3 model in 53% of cases.
Discussion and Future Work
A key insight from this research is the "collapsing attention" phenomenon, where distinct attention regions for relations merge during generation; R-Bind effectively prevents this collapse. However, the method introduces computational overhead, increasing generation time on SD-1.5 from approximately 2 to 11 seconds. Future research will focus on optimizing the step size ($\alpha$) to improve efficiency and stability in complex models like SD-3.
Summary updated