Adapting the Sample Size in Texture Synthesis

Roberto Costantini, Luciano Sbaiz, Sabine SüsstrunkInfoscience 2005

Abstract — Starting from a sample of a given size, texture synthesis algorithms are used to create larger texture images. A good algorithm produces synthesized textures that are pixelwise different but perceptually indistinguishable from the original image. The sample image should be chosen ensuring that it contains a number of pattern repetitions sufficient to produce valuable synthesis results. Since textures can be characterized by patterns of different dimensions, this must be done in an adaptive way. In this article, we propose a method that automatically adapts the sample size for natural textures synthesis, according to the different patterns dimensions. The method is based on the measure of the spatial dependence between the texture pixel values. This measure is used to estimate the size of the smallest texture window that is still perceived as texture by human observers. The sample size is determined from this measure by applying a multiplicative factor that depends on the algorithm used for synthesis. We perform a simple subjective experiment to estimate this factor for three different synthesis algorithms. We show that the measure of spatial dependence based on the correlation between pixels performs well when it is used to adapt the sample size.

The "Goldilocks" Problem in Texture Synthesis

Texture synthesis is the process of taking a small sample image and generating a larger version that looks like the same material but isn't a simple tiled repetition. For this to work, the input sample must be "representative"—it needs to contain enough repetitions of the underlying pattern (often called "textons") for the algorithm to learn the rules of the texture.

Conceptual illustration of varying window sizes ($w_1, w_2$) relative to a fixed screen window ($w$) to determine the minimum perceptive scale.

Historically, researchers chose sample sizes manually or used arbitrary defaults. If the sample is too small, the synthesis fails to capture the texture's essence; if it’s too large, the process becomes computationally expensive and wasteful. The challenge is that "enough" depends entirely on the texture itself—a grain of sand requires a much smaller window than a pile of large river stones. Roberto Costantini and Sabine Süsstrunk propose an automated method to determine this minimum sample size by measuring spatial dependencies between pixels.

Methodology: From Pixels to Perception

The researchers developed a two-step framework to automate this selection:

1. Estimating Perceptive Scale ($W^P_{min}$)

They first identify the smallest crop of a texture that a human still recognizes as that texture. To do this automatically, they tested three statistical measures of spatial dependence to see which best correlates with human perception:

  • Moran’s I statistic: A measure of spatial autocorrelation (how much a pixel looks like its neighbors).
  • Mutual Information: An information-theoretic measure of how much knowing one pixel tells you about another at a certain distance.
  • Random Walk/Run-up test: A statistical test for independence based on scanning the image in a random path.
Comparison of three spatial dependence measures: Moran’s I (a), Mutual Information (b), and Random Walk (c) as a function of pixel distance.

2. The Ideal Observer Model and Scaling

They created a mathematical function that maps these pixel statistics to the probability that a human would perceive the window as a valid texture. Since different algorithms have different "appetites" for data, they introduced a multiplicative factor ($\gamma$). The final sample size ($W^S_{min}$) is calculated as $\gamma \cdot W^P_{min}$. They estimated this factor by running subjective experiments on three popular algorithms: Simoncelli & Portilla (model-based), Ashikhmin (patch-based), and Zelinka (fast patch-based).

Key Insights and Results

  • Simplicity Wins: Surprisingly, the Moran’s I statistic—the simplest of the three measures—performed just as well as the more complex mutual information approach. This suggests that simple linear correlation is a robust enough proxy for texture scale.
  • The "Algorithm Score": The study quantifies how efficient different algorithms are. For example, the Ashikhmin algorithm (ASH) often required a sample size only half as large as the perceptive minimum ($\gamma \approx 0.5$) to produce a convincing result, whereas the model-based Portilla algorithm (POR) often needed much more data for structured textures.
  • Predictive Accuracy: The model's predictions of human responses reached a 98% statistical equivalence in their validation tests. In qualitative tests, the system successfully adapted sample sizes across vastly different scales, correctly identifying that a "Beans" texture required a much larger pixel window than a "Wall" texture.

Conclusion

By bridging objective pixel math with subjective human experience, this framework allows computers to "guess" when a human would stop seeing a texture and start seeing just a collection of random shapes. Using the calculated $W^S_{min}$, researchers can avoid using oversized samples, significantly speeding up synthesis without sacrificing visual quality. While primarily tested on natural textures, future work could extend these "algorithm scores" to modern deep-learning-based synthesis methods.

Summary updated

Abstract

Starting from a sample of a given size, texture synthesis algorithms are used to create larger texture images. A good algorithm produces synthesized textures that are pixelwise different but perceptually indistinguishable from the original image. The sample image should be chosen ensuring that it contains a number of pattern repetitions sufficient to produce valuable synthesis results. Since textures can be characterized by patterns of different dimensions, this must be done in an adaptive way. In this article, we propose a method that automatically adapts the sample size for natural textures synthesis, according to the different patterns dimensions. The method is based on the measure of the spatial dependence between the texture pixel values. This measure is used to estimate the size of the smallest texture window that is still perceived as texture by human observers. The sample size is determined from this measure by applying a multiplicative factor that depends on the algorithm used for synthesis. We perform a simple subjective experiment to estimate this factor for three different synthesis algorithms. We show that the measure of spatial dependence based on the correlation between pixels performs well when it is used to adapt the sample size. I.
BibTeX
@inproceedings{costantini2005adapting,
  author = {Roberto Costantini and Luciano Sbaiz and Sabine Süsstrunk},
  title = {{Adapting the Sample Size in Texture Synthesis}},
  booktitle = {Infoscience},
  year = {2005},
}
Figure