FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RL
Lists
Only you see this
Abstract
Recent studies extend the autoregression paradigm to text-to-image generation, achieving performance comparable to diffusion models. However, our new PairComp benchmark -- featuring test cases of paired prompts with similar syntax but different fine-grained semantics -- reveals that existing models struggle with fine-grained text-image alignment, thus failing to realize precise control over visual tokens. To address this, we propose FocusDiff, which enhances fine-grained text-image semantic alignment by focusing on subtle differences between similar text-image pairs. We construct a new dataset of paired texts and images with similar overall expressions but distinct local semantics, further introducing an improved GRPO-based algorithm to emphasize such fine-grained semantic differences for desired image generation. Our approach achieves superior performance on existing text-to-image benchmarks and also outperforms most prior prominent methods on PairComp. Anonymous Project: https://anonymous.4open.science/r/FocusDiff_Anonym-1F44.
BibTeX
@misc{pan2025focusdiff,
author = {Kaihang Pan and Wendong Bu and Yuruo Wu and Kai Shen and Yang Wu and Yun Zhu and Zehan Wang and Yunfei Li and ZhaoHang and Juncheng Li and Siliang Tang},
title = {{FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RL}},
year = {2025},
eprint = {2506.05501},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2506.05501},
}