STD-Former: Image-Conditioned Texture Dictionary Encoding with Sparse Topological Supervision for Texture Recognition

Bo Peng, Ke Xu, Yurui PanICML 2026 · OpenReview

Texture recognition—the ability to identify materials like wood, fabric, or stone based on repetitive visual patterns—is a fundamental challenge in computer vision. Traditionally, models approach this by learning a static codebook (a dictionary of "textons") during training. At test time, the model attempts to match the features of a new image to this fixed dictionary.

However, this assumption is brittle in real-world scenarios. Label-preserving transformations such as varying illumination, scale changes, compression, or motion blur can shift an image's features away from the static training dictionary. The authors of this paper identify this phenomenon as Training-Set Codebook Misalignment. If the fixed dictionary fails to account for how a texture appears under poor lighting or blur, classification accuracy plummets.

To solve this, the authors introduce STD-Former (Simple Texture Dictionary Transformer), a lightweight framework that abandons static codebooks in favor of image-conditioned texture dictionary encoding.

Introducing STD-Former: Image-Conditioned Dictionaries

Instead of comparing an input image against a pre-learned, static set of codewords, STD-Former generates a custom dictionary for every single image it processes. By extracting a compact set of Intrinsic Textons directly from the input image, the model yields self-aligned representations at inference time. This allows the model to adapt dynamically to the specific conditions (e.g., blur, lighting) of the input, bypassing the need for the test image to perfectly match a training-time average.

A Decoupled Two-Stage Architecture

STD-Former relies on a powerful backbone (such as DINOv2 or ResNet) to extract patch features, which are then processed through a decoupled two-stage recipe:

  1. Stage 1: Self-Supervised Dictionary Extraction. The core of the model is the Texture Dictionary Extractor (TDE). Using a cross-attention mechanism, learnable queries interact with the image's features to distill the Intrinsic Textons. The TDE is pre-trained using a Texton Coverage Loss, which forces the extracted textons to collectively cover the image patch feature manifold. This ensures the textons are truly representative of the underlying patterns without requiring class labels.
  2. Stage 2: Supervised Classification. A lightweight classifier is trained on the encoded dictionary representation. By decoupling "what a texture looks like" (Stage 1) from "what class it belongs to" (Stage 2), the model learns highly transferable and robust features.

Sparse Topological Supervision

For the enhanced version of the model, STD-Former+, the authors introduce a Sparse Topological Loss. Traditional topological losses can be computationally heavy. However, the authors leverage 0D persistent homology, proving it is equivalent to supervising only the $(B-1)$ edges of a Minimum Spanning Tree (MST) in each batch.

Instead of comparing every image to every other image, this sparse supervision focuses only on the most critical relationships in a batch. It pushes images of the same class together and pulls different classes apart along these critical edges, creating a highly organized and discriminative feature space with negligible computational overhead.

Datasets and Evaluation

The researchers evaluated STD-Former across six diverse, standard texture benchmarks to ensure robust generalization:

  • DTD (Describable Textures Database): 47 categories of "wild" textures.
  • FMD (Flickr Material Dataset): 10 common material categories.
  • MINC (Materials in Context): A large-scale dataset for materials in real-world scenes.
  • GTOS (Ground Terrain in Outdoor Scenes): 40 outdoor ground materials.
  • Fabrics: Fine-grained material classification.
  • KTH-TIPS2b: Textures captured under varying scales, lighting, and poses.

Key Results and Performance

STD-Former and STD-Former+ establish new state-of-the-art (SOTA) results across all tested benchmarks:

  • Unprecedented Accuracy: On the notoriously challenging DTD dataset, STD-Former+ achieved 86.1% accuracy, representing a massive 7.0% improvement over previous leading methods like GraphTEN.
  • Robustness to Misalignment: In stress tests simulating blur, rotation, and compression, STD-Former exhibited significantly less performance degradation compared to models relying on static codebooks.
  • Computational Efficiency: Despite its sophisticated topological math, the model remains lightweight. The ResNet-50 variant processes an image in just 17.2ms, making it highly practical for real-world, real-time deployment.

Limitations and Future Directions

While highly effective, the framework does have a few limitations. First, its most impressive performance gains rely heavily on the high-quality features provided by the DINOv2 backbone. Second, the Sparse Topological Loss is batch-sensitive; it requires class-balanced batches to function optimally, as batches containing only a single class render the inter-class separation penalty inactive.

Looking forward, the authors suggest that the core concept of an "image-conditioned dictionary" extends far beyond textures. This self-aligning approach holds significant promise for other fine-grained visual recognition tasks—such as medical imaging analysis or plant disease diagnosis—where local, instance-specific structures are often more critical than global object shapes.

Summary updated

Abstract

Texture recognition is often framed as matching an image to a static training-set dictionary or codebook. In practice, this assumption is brittle: label-preserving transformations (illumination, scale, compression, blur) can shift test features away from the fixed training dictionary, producing a training-set codebook misalignment that limits accuracy. We propose STD-Former (Simple Texture Dictionary Transformer), a lightweight framework for image-conditioned texture dictionary encoding. Instead of comparing against a static codebook, STD-Former extracts a compact set of Intrinsic Textons (dictionary atoms / codewords) from the input image itself, yielding self-aligned representations at inference. Our design is intentionally simple and uses a decoupled two-stage recipe. In Stage 1, a Texture Dictionary Extractor (TDE) is pre-trained with a self-supervised Texton Coverage Loss that encourages the learned textons to collectively cover the image patch feature manifold. In Stage 2, a classifier is trained on the encoded dictionary representation; optionally, we add a Sparse Topological Loss derived from 0D persistent homology, which is equivalent to supervising only the (B-1) edges of a minimum spanning tree (MST) in each batch, providing efficient structure regularization. Across six standard texture benchmarks, STD-Former and STD-Former+ achieve new state-of-the-art results.
BibTeX
@inproceedings{peng2026std,
  author = {Bo Peng and Ke Xu and Yurui Pan},
  title = {{STD-Former: Image-Conditioned Texture Dictionary Encoding with Sparse Topological Supervision for Texture Recognition}},
  booktitle = {International Conference on Machine Learning},
  year = {2026},
  url = {https://openreview.net/forum?id=wgOItdKqUS},
}
Figure