Abstract
This paper addresses the problem of interpolating visual textures. We formulate this problem by requiring (1) by-example controllability and (2) realistic and smooth interpolation among an arbitrary number of texture samples. To solve it we propose a neural network trained simultaneously on a reconstruction task and a generation task, which can project texture examples onto a latent space where they can be linearly interpolated and projected back onto the image domain, thus ensuring both intuitive control and realistic results. We show our method outperforms a number of baselines according to a comprehensive suite of metrics as well as a user study. We further show several applications based on our technique, which include texture brush, texture dissolve, and animal hybridization.
Introduction
Interpolating between textures—for instance, transitioning from leopard spots to tiger stripes—poses a significant challenge in computer vision. Traditional methods like $\alpha$-blending often result in "ghosting" artifacts, where patterns appear semi-transparently overlaid. Conversely, stitching methods frequently produce visible, unnatural seams. While patch-based synthesis can yield higher quality, it is computationally intensive and lacks the "by-example" controllability required for creative applications.
Texture Mixer introduces a feedforward neural network designed to map textures into a specialized latent space. Within this space, linear interpolation corresponds to perceptually smooth transitions, enabling the synthesis of realistic, hybrid textures at interactive speeds.
Methodology
The Texture Mixer architecture is built around a dual-task training regime: reconstruction and interpolation.
Dual Encoding and Latent Space
The system employs two distinct encoders to capture the essence of a texture. A global encoder ($E^g$) compresses the overall style into a $1 \times 1$ latent vector, while a local encoder ($E^l$) preserves spatial structure in a downsampled latent tensor. By interpolating these latent representations rather than raw pixels, the model avoids the pitfalls of naïve blending.
Random Shuffling
A key innovation is the random shuffling operation applied to latent tensors. To prevent the generator from simply memorizing and repeating patterns—which leads to artificial "tiling"—the network swaps adjacent small squares in the latent space. This regularizer forces the model to learn the underlying statistics of a texture, ensuring the output remains stochastic and natural.
Experiments and Results
To evaluate the model, the authors curated three high-quality datasets: Earth Texture (896 images), Animal Texture (866 images), and Plant Texture (1,074 images). These real-world RGB photos provide a more rigorous benchmark than standard synthetic datasets.
Performance and User Study
Texture Mixer achieves high-quality results in approximately 0.5 seconds, significantly faster than patch-based methods like Image Melding, which can take several minutes. In a large-scale user study on Amazon Mechanical Turk, participants consistently preferred Texture Mixer over baselines such as WCT and PSGAN. The model balances controllability, smoothness, and realism, as evidenced by its superior performance across a suite of metrics including Center Seam and Center Repetition scores.
Applications
The framework enables several creative tools, most notably the "Texture Brush." This tool allows artists to sample specific regions of a source texture and paint with an interpolated palette, as demonstrated in the synthesis of the "CVPR" text below.
Conclusion and Limitations
While Texture Mixer excels at stochastic, natural textures, it faces limitations with strong structural textures. Rigid geometries, such as brick walls or honeycombs, can become warped or "melted" during latent shuffling and interpolation. Future research may explore incorporating geometric constraints to better preserve these structured patterns while maintaining the benefits of neural interpolation.
Summary updated