Abstract
Researchers from Peking University have developed a unified framework for interactive texture transfer that excels at preserving both complex local structures and fine-grained visual details. By combining user-defined semantic maps with automatically extracted "structure channels," the system can turn doodles into masterpieces, generate stylized text, and swap textures between objects in just a few minutes—significantly faster than recent deep-learning alternatives.
Introduction
Texture transfer is the art of taking the "look and feel" of a source image and applying it to a target. While the field has seen a surge in neural-network-based approaches, two major hurdles remain: task isolation and structural collapse. Most algorithms are designed specifically for one type of input (e.g., text or doodles), and they often lose fine internal lines of the source—such as delicate brush strokes or veins—leading to "washed-out" results. This research introduces a general-purpose solution using "multi-channel dynamic guidance" to ensure the structural integrity of the source texture is preserved during the transfer process.
Methodology
The framework operates through a three-stage pipeline:
- Internal Salient Structure Extraction: The system automatically identifies "salient" regions in the source image (edges, contours, and high-contrast textures) that are not captured by the user's simple semantic map.
- Structure Propagation: To map these source structures onto the target, the authors use Coherent Point Drift (CPD) to match contours and Thin Plate Splines (TPS) to warp the structural information into the new shape. This creates a "prior" that tells the synthesis engine where the important lines should go.
- Guided Texture Transfer: The final image is synthesized using an optimization-based approach. The authors improved the classic PatchMatch algorithm by extending the Nearest Neighbor Field (NNF) to allow for rotations and reflections, and by implementing dynamic weighting to prioritize layout initially and coherence later.
Key Results and Versatility
The method demonstrates significant improvements in both visual fidelity and computational efficiency. It successfully preserves "fragile" textures where CNN-based methods often produce blurry artifacts.
- Speed: The framework synthesizes a 500x400 pixel image in approximately 2 minutes on a standard CPU, compared to 15 minutes for Deep Image Analogy and 40 minutes for Neural Doodle.
- Versatility: The authors demonstrate success across five distinct tasks, including doodle-to-artwork, decorative pattern editing, and texture swapping.
Conclusion
While the framework performs best when source and target shapes are relatively similar, it offers a robust and efficient alternative to heavy convolutional neural networks. Future work aims to leverage GPU acceleration to achieve real-time synthesis, further enhancing its utility in interactive design workflows.
Summary updated