Abstract
Texture synthesis aims to generate large, seamless images from small source examples. While traditional Markov Random Field (MRF) methods preserve structure well, they often lack the flexibility of neural approaches. Conversely, neural methods frequently produce blurry or repetitive results. This paper introduces a "Guided Correspondence" framework that integrates MRF-style patch similarity with CNN representations. By employing a Guided Correspondence Distance (GCD) and a specialized loss function ($L_{GC}$), the method eliminates common artifacts and enables real-time, user-controlled texture generation.
Introduction
The challenge of texture synthesis lies in maintaining both local detail and global structure. Traditional methods based on Markov Random Fields (MRFs) typically "copy-paste" patches, which preserves sharpness but can fail on complex semantic structures. Modern neural methods, such as CNNMRF, often result in blurry outputs due to the "averaging" effect of Euclidean distance optimization in feature space. Furthermore, these methods frequently suffer from unnatural repetition when the algorithm over-uses a single "best-fit" source patch.
Guided Correspondence Framework
The proposed framework addresses these limitations through two primary innovations: the Guided Correspondence Distance (GCD) and the Guided Correspondence Loss ($L_{GC}$).
Guided Correspondence Distance (GCD)
To prevent the repetitive patterns common in neural synthesis, the authors introduce an occurrence penalty. This mechanism tracks the frequency of source patch usage; as a patch is used more often, its perceived distance increases, forcing the selection of diverse alternatives. Additionally, guidance channels allow users to incorporate spatial constraints, such as orientation fields or semantic masks, directly into the patch-matching process.
Guided Correspondence Loss ($L_{GC}$)
The core of the method's ability to produce sharp textures is the $L_{GC}$. Unlike standard $L_2$ losses that minimize the average distance to all similar patches, $L_{GC}$ employs a "winner-takes-all" approach inspired by contextual loss. It requires a target patch to be significantly closer to its single best-match source patch than to any others. This prevents the optimization from "hedging its bets" and producing blurry averages.
Implementation and Results
The framework is versatile, supporting both high-quality iterative optimization and real-time synthesis when used as a loss function to train Generative Adversarial Networks (GANs). The system utilizes a pre-trained VGG-19 network as a feature extractor to calculate correspondence distances.
Performance and Quality
Evaluated on a dataset of 50 diverse textures, the method consistently outperforms prior state-of-the-art techniques like SinGAN and CNNMRF. Qualitative results show significantly sharper details and better structural coherence. In a user study involving 40 participants and 2,000 questions, the proposed method was preferred by a large margin over other neural approaches.
User Control
One of the standout features is the degree of interactive control. Users can guide the synthesis process using orientation maps (e.g., for wood grain) or progression maps (e.g., for growing biological textures). While optimization-based synthesis offers the highest fidelity (taking 5–20 minutes), the GAN-based approach allows for these edits in real-time via a single forward pass.
Conclusion
By rethinking patch correspondence through the lens of diversity and competitive similarity, this work successfully bridges the gap between classical patch-based synthesis and modern neural networks. The Guided Correspondence framework provides a robust tool for high-quality, controllable texture generation suitable for both offline rendering and interactive design.
Summary updated