Abstract
Texture synthesis is a fundamental task in computer graphics, yet the preparation of complex, non-stationary natural textures remains a labor-intensive process. This paper introduces a fully unsupervised, multi-scale framework for automatically extracting "label-maps" from natural images to guide synthesis algorithms. By integrating a novel hierarchical superpixel decomposition (H-SLIC) with non-negative matrix factorization (NMF), the proposed method eliminates the need for manual annotation. The system effectively captures texture structures across multiple scales and automatically determines the optimal number of labels, facilitating the synthesis of high-quality, complex textures from natural exemplars.
Introduction
Texture synthesis involves generating a large-scale image from a small example while maintaining visual consistency. While stationary textures—those with repeating patterns—are well-handled by existing algorithms, many "natural" textures are non-stationary. For instance, a mossy stone wall or a forest floor contains a non-uniform distribution of elements like moss, stones, and cracks that change across the surface.
To synthesize such textures effectively, algorithms require "guidance maps" or label-maps that identify the spatial distribution of different patterns. Traditionally, creating these maps is an "extremely tedious" manual task for artists. Furthermore, natural textures exhibit hierarchical structures—such as a forest having both "tree" and "leaf" scales—which automated tools often fail to capture without significant user intervention or prior knowledge of the number of patterns.
Methods
The authors propose a two-stage, fully automatic pipeline for label-map extraction that requires zero prior training or user input:
Hierarchical Partitioning (H-SLIC)
Standard superpixel algorithms, such as SLIC, often blur important edges when operating at coarse scales. To address this, the authors developed H-SLIC, a hierarchical extension that begins with fine-grained superpixels and iteratively merges them. This approach ensures that large-scale regions maintain the sharp boundaries established by fine-scale details, preserving the structural integrity of the texture across levels.
Unsupervised Labeling via NMF
After partitioning, the algorithm must group similar regions. This is modeled as a random walk problem on a bipartite graph, where superpixels are connected to abstract "cluster nodes." The system employs Non-negative Matrix Factorization (NMF) to identify these latent clusters.
Unlike traditional clustering methods like K-means, this NMF-based approach automatically determines the optimal number of labels and the appropriate scales for analysis. By using the eigenvalues of the similarity matrix, the system mathematically calculates the necessary "diffusion steps" to capture the texture's structure, removing the need for manual parameter tuning.
Results and Insights
The method was evaluated on a new benchmark of 15 high-resolution natural images and standard texture sets. Key findings include:
- Boundary Accuracy: H-SLIC demonstrated significantly lower under-segmentation error compared to standard hierarchical methods, more accurately capturing true object boundaries.
- Robustness: The use of a bipartite graph makes the system resilient to noise and efficient at handling large-scale image data. By connecting superpixels to abstract cluster nodes rather than directly to each other, the system handles sparse data more effectively.
- Scale-Dependent Stationarity: The algorithm explicitly recognizes that patterns may be stationary at one scale but unique objects at another. This allows it to label sub-patterns within larger structures accurately.
- Performance: The system scales to megapixel images. A 2.5-megapixel image can be partitioned in approximately one minute, though the complex multi-scale labeling phase can take up to 45 minutes for very large files.
Conclusion
This work provides a robust, unsupervised solution for the automated analysis of non-stationary textures. By extracting hierarchical label-maps, it significantly reduces the manual overhead in texture synthesis workflows. While current features rely on color moments, future work could incorporate geometric features like Gabor filters or deep-learning-based descriptors to capture more complex patterns like bricks. Additionally, integrating these label-maps directly into the "seam cutting" logic of patch-based synthesis algorithms holds promise for even more seamless results in complex natural scenes.
Summary updated