Interactive Sketch & Fill: Multiclass Sketch-to-Image Translation

Arnab Ghosh, Richard Zhang, Puneet K. Dokania, Oliver Wang, Alexei A. Efros, Philip H. S. Torr, Eli Shechtman

Abstract

We propose an interactive GAN-based sketch-to-image translation method that helps novice users create images of simple objects. As the user starts to draw a sketch of a desired object type, the network interactively recommends plausible completions, and shows a corresponding synthesized image to the user. This enables a feedback loop, where the user can edit their sketch based on the network's recommendations, visualizing both the completed shape and final rendered image while they draw. In order to use a single trained model across a wide array of object classes, we introduce a gating-based approach for class conditioning, which allows us to generate distinct classes without feature mixing, from a single generator network. Video available at our website: https://arnabgho.github.io/iSketchNFill/.
BibTeX
@inproceedings{ghosh2019interactive,
  author = {Arnab Ghosh and Richard Zhang and Puneet K. Dokania and Oliver Wang and Alexei A. Efros and Philip H. S. Torr and Eli Shechtman},
  title = {{Interactive Sketch \& Fill: Multiclass Sketch-to-Image Translation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  url = {https://openreview.net/forum?id=INQHaiv5KIo},
}

Loading related content...

Figure