Abstract
This paper introduces Texture Mixer, a feedforward neural network designed for the smooth interpolation of visual textures. By formulating the problem with a focus on by-example controllability and realistic transitions, the authors propose a system trained simultaneously on reconstruction and generation. The model projects texture samples into a shared latent space where linear interpolation yields realistic, artifact-free results. Evaluated against a suite of metrics and user studies, Texture Mixer demonstrates superior performance and speed, enabling applications like texture brushes and animal hybridization.
Introduction
In digital content creation, transitioning between materials—such as blending grass into dirt or merging different animal furs—is a frequent requirement. Traditional methods often fall short: "alpha blending" results in "ghosting" artifacts, while non-parametric patch-based synthesis is computationally expensive and often fails to capture complex natural statistics. While Generative Adversarial Networks (GANs) offer high-quality synthesis, they typically lack the fine-grained controllability needed for specific artistic placement.
Methodology
The Texture Mixer architecture is designed to bridge the gap between realism and control. It is trained on two concurrent objectives: a reconstruction task to ensure fidelity to input samples, and an interpolation task to ensure the latent space between samples represents valid textures.
Dual-Encoder Architecture
The system utilizes two distinct encoders to decompose texture information:
- Global Encoder ($E^g$): Extracts a $1 \times 1$ vector representing the overall style and statistical properties.
- Local Encoder ($E^l$): Produces a latent tensor that captures spatial structures at a reduced resolution.
Latent Shuffling
A key innovation is the random shuffling operation applied within the latent space. By swapping adjacent squares of the latent tensor at multiple scales, the network breaks up repetitive "tiling" patterns. This stochasticity ensures that the final generator ($G$) produces natural-looking textures even when covering large areas.
Datasets and Training
To support this research, the authors curated two high-density datasets:
- Earth Texture Dataset: 994 images of natural terrains like soil and gravel.
- Animal Texture Dataset: 961 images of diverse animal furs and skins.
These datasets provide the necessary variety to train a robust interpolation manifold, allowing the network to learn the underlying logic of texture transitions.
Results and Applications
Texture Mixer achieves high-quality results that avoid the common pitfalls of previous methods. Quantitatively, it outperforms baselines like PSGAN and AdaIN across metrics for controllability and realism. In a user study, participants preferred Texture Mixer's results with rates often exceeding 80%.
Beyond simple blending, the model supports advanced applications. Because it is a feedforward network, it can generate $1024 \times 1024$ textures in roughly 0.5 seconds. This speed enables interactive tools like "texture brushes" and "texture dissolve" effects.
Conclusion
Texture Mixer provides a powerful framework for interpolating textures by mapping them to a continuous, controllable manifold. While currently trained on specific categories, it sets a new standard for speed and quality in texture synthesis. Future work may extend this approach to video textures and 3D surface properties, further expanding the toolkit for digital artists.
Summary updated