Robust tile-based texture synthesis using artificial immune system

Weiming Dong, Ning Zhou, Jean-Claude PaulNeural Computing and Applications 2009 · DOI

TL;DR

This paper introduces a novel approach to texture synthesis using an Artificial Immune System (AIS) to robustly select sample patches for creating tileable textures. By optimizing the quality of the entire tile set, their method effectively reduces conspicuous seams and generates high-quality, diverse textures that can be synthesized in real-time.

What Problem Does This Paper Address?

Generating realistic, tileable textures from small input examples is a significant challenge in computer graphics. A primary issue is the appearance of noticeable seams when smaller texture patches (tiles) are combined to create larger images. This often arises from poorly chosen sample patches that don't stitch together seamlessly. Existing methods, while improving, either struggle with computational cost, produce repetitive patterns, or lack robustness in selecting optimal sample patches, leading to suboptimal texture quality. This paper aims to address the problem of selecting a high-quality, diverse set of sample patches that can be used to construct tileable textures with minimal artifacts, especially for real-time applications.

Methods

The core of this paper is the application of an Artificial Immune System (AIS), specifically a clonal selection algorithm, to the problem of selecting optimal sample patches for texture synthesis.

  • Tile Set Formation: The authors build upon the concept of "w-tiles," which are square blocks with specific colors at their corners, derived from sample patches. They propose a method to increase the number of w-tiles by systematically replacing colors in a basic set, thereby enriching the pattern diversity and reducing repetition.
  • Artificial Immune System (AIS) for Sample Patch Selection:
    • Antibody Representation: Each "antibody" in the AIS represents a candidate set of sample patches.
  • Evaluation Function: A key innovation is the development of an evaluation function that considers both the minimum matching error between an intermediate tile and its best matching patch (local quality) and the variance of these errors across the entire tile set (global quality). This function aims to balance the quality of all tiles.
  • Clonal Selection Algorithm: The AIS uses a clonal selection process. Antibodies with higher "affinity" (better evaluation function scores) are selected to reproduce (clone). These clones then undergo mutation, mimicking the natural immune system's adaptation.
  • Mutation Regulation: The mutation rate is inversely proportional to an antibody's affinity, meaning better solutions are less likely to be drastically altered.
  • Memory Unit: A portion of the population is designated as a "memory unit" to retain good solutions.
  • Graph-Cut for Junction Elimination: After sample patches are selected by the AIS, graph-cut is used to merge the selected patches into intermediate tiles, effectively eliminating seams. Poisson smoothing is also applied to further refine the tiles.

Datasets

The paper uses various real-world texture images for evaluation. Specific examples include:

  • Beans
  • Yellow leaves
  • Tree barks
  • Caustics
  • Stones
  • Bread
  • Grape
  • Grass
  • School
  • Flowers
  • Tomatoes
  • Fabric
  • Bricks
  • Eggs

These are standard texture datasets commonly used in computer graphics research. No new benchmarks are introduced.

Key Insights and Ideas

  • Global Optimization for Tile Sets: The most significant insight is that optimizing the entire set of tiles, rather than just individual tiles, leads to more robust and less repetitive textures. The AIS framework allows for this global optimization.
  • Balancing Local and Global Quality: The evaluation function's dual nature (minimum error and variance) is crucial. It prevents scenarios where a few perfect tiles are achieved at the expense of many poor ones, ensuring overall consistency.
  • AIS as an Effective Selector: The paper demonstrates that AIS, particularly clonal selection, is a powerful and efficient mechanism for solving the complex combinatorial optimization problem of selecting sample patches, outperforming traditional methods like Genetic Algorithms in avoiding local optima.
  • Enriching Tile Sets: The method for systematically increasing the number of w-tiles by adding new color combinations is a practical way to enhance pattern diversity and reduce visual repetition.

Key Results

  • High-Quality Texture Synthesis: The proposed method generates visually pleasing, tileable textures with significantly reduced seam artifacts compared to previous approaches.
  • Real-time Performance: While the AIS-based sample patch selection is a pre-computation step, the actual texture tiling process is performed in real-time, making it suitable for interactive applications.
  • Efficient Selection: The AIS-based selection process is shown to be efficient, with timings for various examples listed in Table 1, generally ranging from under a minute to a few minutes for pre-computation.
  • Robustness: The method demonstrates robustness across a variety of texture types, performing particularly well on non-structural and semi-structural textures.

Limitations and Future Work

  • Structural Textures: The authors acknowledge that the method exhibits limitations with highly structural textures (e.g., the "eggs" example), where artifacts can still be apparent. This is attributed to the constraint of using a single patch to fill junctions.
  • Future Work:
    • Extending the technique to handle geometric textures on 3D models.
  • Exploring other local region-growing methods (like texture optimization) for tile construction to improve the synthesis of structural textures.
  • Investigating the use of other distance metrics beyond color, such as image gradients, to capture more texture characteristics.

Summary updated

BibTeX
@article{dong2009robust,
  author = {Weiming Dong and Ning Zhou and Jean-Claude Paul},
  title = {{Robust tile-based texture synthesis using artificial immune system}},
  journal = {Neural Computing and Applications},
  year = {2009},
  doi = {10.1007/s00521-008-0174-6},
}
Figure