L-CoIns: Language-based Colorization With Instance Awareness
Lists
Only you see this
Abstract
Language-based colorization produces plausible colors consistent with the language description provided by the user. Recent studies introduce additional annotation to prevent color-object coupling and mismatch issues, but they still have difficulty in distinguishing instances corresponding to the same object words. In this paper, we propose a transformer-based framework to automatically aggregate similar image patches and achieve instance awareness without any additional knowledge. By applying our presented luminance augmentation and counter-color loss to break down the statistical correlation between luminance and color words, our model is driven to synthesize colors with better descriptive consistency. We further collect a dataset to provide distinctive visual characteristics and detailed language descriptions for multiple instances in the same image. Extensive experiments demonstrate our advantages of synthesizing visually pleasing and description-consistent results of instance-aware colorization.
BibTeX
@inproceedings{chang2023l,
author = {Zheng Chang and Shuchen Weng and Peixuan Zhang and Yu Li and Si Li and Boxin Shi},
title = {{L-CoIns: Language-based Colorization With Instance Awareness}},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2023},
doi = {10.1109/cvpr52729.2023.01842},
}Loading related content...