Diffusion-Enhanced PatchMatch: A Framework for Arbitrary Style Transfer with Diffusion Models

Mark Hamazaspyan, Shant NavasardyanWorkshops @ CVPR 2023 · OpenReview

Abstract

Diffusion models have gained immense popularity in recent years due to their impressive ability to generate high-quality images. The opportunities that diffusion models provide are numerous, from text-to-image synthesis to image restoration and enhancement, as well as image compression and inpainting. However, expressing image style in words can be a challenging task, making it difficult for diffusion models to generate images with specific style without additional optimization techniques. In this paper, we present a novel method, Diffusion-Enhanced PatchMatch (DEPM), that leverages Stable Diffusion for style transfer without any finetuning or pretraining. DEPM captures high-level style features while preserving the fine-grained texture details of the original image. By enabling the transfer of arbitrary styles during inference, our approach makes the process more flexible and efficient. Moreover, its optimization-free nature makes it accessible to a wide range of users.
BibTeX
@inproceedings{hamazaspyan2023diffusion,
  author = {Mark Hamazaspyan and Shant Navasardyan},
  title = {{Diffusion-Enhanced PatchMatch: A Framework for Arbitrary Style Transfer with Diffusion Models}},
  booktitle = {Workshops @ CVPR},
  year = {2023},
  url = {https://openreview.net/forum?id=OALwBEXJbc},
}

Loading related content...

Figure