A Common Framework for Interactive Texture Transfer

Yifang Men, Zhouhui Lian, Yingmin Tang, Jianguo XiaoCVPR 2018 · DOI

Abstract

Researchers from Peking University have developed a unified framework for interactive texture transfer that excels at preserving both complex local structures and fine-grained visual details. By combining user-defined semantic maps with automatically extracted "structure channels," the system can turn doodles into masterpieces, generate stylized text, and swap textures between objects in just a few minutes—significantly faster than recent deep-learning alternatives.

Introduction

Texture transfer is the art of taking the "look and feel" of a source image and applying it to a target. While the field has seen a surge in neural-network-based approaches, two major hurdles remain: task isolation and structural collapse. Most algorithms are designed specifically for one type of input (e.g., text or doodles), and they often lose fine internal lines of the source—such as delicate brush strokes or veins—leading to "washed-out" results. This research introduces a general-purpose solution using "multi-channel dynamic guidance" to ensure the structural integrity of the source texture is preserved during the transfer process.

Methodology

The framework operates through a three-stage pipeline:

  1. Internal Salient Structure Extraction: The system automatically identifies "salient" regions in the source image (edges, contours, and high-contrast textures) that are not captured by the user's simple semantic map.
  2. Structure Propagation: To map these source structures onto the target, the authors use Coherent Point Drift (CPD) to match contours and Thin Plate Splines (TPS) to warp the structural information into the new shape. This creates a "prior" that tells the synthesis engine where the important lines should go.
  3. Guided Texture Transfer: The final image is synthesized using an optimization-based approach. The authors improved the classic PatchMatch algorithm by extending the Nearest Neighbor Field (NNF) to allow for rotations and reflections, and by implementing dynamic weighting to prioritize layout initially and coherence later.
Example of special effect text generation using the proposed framework.

Key Results and Versatility

The method demonstrates significant improvements in both visual fidelity and computational efficiency. It successfully preserves "fragile" textures where CNN-based methods often produce blurry artifacts.

  • Speed: The framework synthesizes a 500x400 pixel image in approximately 2 minutes on a standard CPU, compared to 15 minutes for Deep Image Analogy and 40 minutes for Neural Doodle.
  • Versatility: The authors demonstrate success across five distinct tasks, including doodle-to-artwork, decorative pattern editing, and texture swapping.
Texture swapping between apples, demonstrating the framework's ability to handle object-to-object transfer with semantic guidance.

Conclusion

While the framework performs best when source and target shapes are relatively similar, it offers a robust and efficient alternative to heavy convolutional neural networks. Future work aims to leverage GPU acceleration to achieve real-time synthesis, further enhancing its utility in interactive design workflows.

Summary updated

BibTeX
@inproceedings{men2018common,
  author = {Yifang Men and Zhouhui Lian and Yingmin Tang and Jianguo Xiao},
  title = {{A Common Framework for Interactive Texture Transfer}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2018},
  doi = {10.1109/cvpr.2018.00665},
}
Figure