Attentive Linguistic Tracking in Diffusion Models for Training-free Text-guided Image Editing
Lists
Only you see this
Abstract
Building on recent breakthroughs in diffusion-based text-to-image synthesis (TIS), training-free text-guided image editing (TIE) has become an indispensable aspect of modern image editing practices. It involves modifying the features in attention layers to alter objects or their attributes within images during the generation process. Yet, current image editing algorithms still present certain difficulties and challenges when it comes to editing multiple objects within an image. In this paper, we propose VICTORIA, a novel approach that enhances TIE by incorporating linguistic knowledge when manipulating attention maps during image generation. VICTORIA leverages components within self-attention layers to maintain spatial consistency between source and target images. Additionally, a novel loss function is designed to refine cross-attention maps, ensuring their alignment with linguistic constraints and enhancing the editing of multiple target entities. We also introduce a linguistic mask blending technique to improve the retention of information in areas exempt from modification. Experimental results across seven diverse datasets demonstrate that VICTORIA achieves substantial enhancements over state-of-the-art methods. This work highlights the critical role and effectiveness of linguistic analysis in boosting the performance of TIE.
BibTeX
@inproceedings{liu2024attentive,
author = {Bingyan Liu and Chengyu Wang and Jun Huang and Kui Jia},
title = {{Attentive Linguistic Tracking in Diffusion Models for Training-free Text-guided Image Editing}},
booktitle = {ACM Multimedia},
year = {2024},
doi = {10.1145/3664647.3680594},
}Loading related content...