SketchTriplet: Self-Supervised Scenarized Sketch-Text-Image Triplet Generation
Lists
Only you see this
Abstract
The scarcity of free-hand sketch presents a challenging problem. Despite the emergence of some large-scale sketch datasets, these datasets primarily consist of sketches at the single-object level. There continues to be a lack of large-scale paired datasets for scene sketches. In this paper, we propose a self-supervised method for scene sketch generation that does not rely on any existing scene sketch, enabling the transformation of single-object sketches into scene sketches. To accomplish this, we introduce a method for vector sketch captioning and sketch semantic expansion. Additionally, we design a sketch generation network that incorporates a fusion of multi-modal perceptual constraints, suitable for application in zero-shot image-to-sketch downstream task, demonstrating state-of-the-art performance through experimental validation. Finally, leveraging our proposed sketch-to-sketch generation method, we contribute a large-scale dataset centered around scene sketches, comprising highly semantically consistent "text-sketch-image" triplets. Our research confirms that this dataset can significantly enhance the capabilities of existing models in sketch-based image retrieval and sketch-controlled image synthesis tasks. We will make our dataset and code publicly available.
BibTeX
@article{wu2025sketchtriplet,
author = {Zhenbei Wu and Qiang Wang and Jie Yang},
title = {{SketchTriplet: Self-Supervised Scenarized Sketch-Text-Image Triplet Generation}},
journal = {IEEE Internet of Things Journal},
year = {2025},
doi = {10.1109/jiot.2024.3523382},
}Loading related content...