Deep Multiple-Attribute-Perceived Network for Real-World Texture Recognition

Wei Zhai, Yang Cao, Jing Zhang, Zheng-Jun Zha

Abstract

Texture recognition in uncontrolled environments is a complex task because real-world surfaces often exhibit multiple overlapping visual properties. Traditional single-label classification models fail to capture this richness, leading to incomplete representations. This paper introduces the Multiple-Attribute-Perceived Network (MAP-Net), a multi-branch architecture designed to progressively learn correlated visual attributes. By cascading features from coarse to fine scales and employing an Attribute Transfer Module (ATM) with deformable pooling, MAP-Net builds a more descriptive and robust texture representation. Experimental results on five major benchmarks—DTD, FMD, KTH-TIPS2b, GTOS, and GTOS-mobile—demonstrate that MAP-Net consistently outperforms existing state-of-the-art methods.

Introduction

Texture recognition is fundamental to computer vision, yet "in the wild" performance remains limited. Most existing models treat textures as mutually exclusive categories (e.g., "brick" vs. "grass"). However, human perception of texture is multi-faceted; a "marble" surface is simultaneously "veined," "cracked," and "marbled." Standard CNNs, optimized for object recognition, often discard local variations that are essential for identifying these coexisting attributes. The authors argue that failing to explicitly model these attributes produces incomplete and often inaccurate descriptions of real-world textures.

Figure 1: (A) Examples of textures with multiple descriptive attributes. (B) Ambiguity in single-label classification where multiple attributes (e.g., "Chequered") can describe different textures.

The MAP-Net Framework

The proposed MAP-Net addresses these challenges through a multi-branch cascading structure that "peels back" layers of texture information.

Multi-Branch Cascading and Attribute Transfer

Instead of a single-pass feature extraction, MAP-Net utilizes multiple branches. The initial branch captures coarse, global features which then guide subsequent branches to focus on finer, more specific attributes. The Attribute Transfer Module (ATM) facilitates this by using global context from previous branches to calculate adaptive offsets for deformable pooling. This allows the network to dynamically shift its receptive field to regions where secondary attributes are most prominent.

Figure 5: (a) Conceptual flow of progressive attribute learning for "Grid" and "Marble" textures. (b) Detailed MAP-Net architecture showing the integration of local features and global context across cascading attribute branches.

Optimization and Fusion

To ensure discriminative power, the network is trained using a combination of standard softmax loss and triplet loss. The triplet loss forces the model to cluster similar textures while separating dissimilar ones in the feature space, making the representation more organized. Finally, a Multi-Attribute Fusion (MAF) module learns optimal weights for each branch's contribution to the final classification, prioritizing the most descriptive attributes detected.

Experiments and Results

MAP-Net was evaluated on five challenging datasets: DTD, FMD, KTH-TIPS2b, GTOS, and GTOS-mobile.

  • DTD (Describable Texture Database): Achieved 76.1% accuracy, a 3% improvement over previous state-of-the-art (SOTA).
  • FMD (Flickr Material Dataset): Reached 85.7% accuracy, significantly surpassing the previous best of ~82%.
  • GTOS-mobile: Achieved 86.64%, demonstrating robustness to real-world noise and blur typical of mobile photography.

Ablation studies confirmed the efficacy of the multi-branch approach. Increasing the number of branches from one to three yielded steady performance gains, validating the hypothesis that progressive attribute learning captures more information than single-label approaches. However, the authors noted diminishing returns after three branches, suggesting a limit to the levels of attributes extractable from standard-resolution images.

Conclusion

MAP-Net represents a shift from categorical to descriptive texture recognition. By explicitly modeling the correlation between visual attributes, the network achieves superior performance and provides a more nuanced understanding of material surfaces. Future work will explore the integration of this multi-attribute engine into broader scene understanding tasks, where textures serve as the fundamental building blocks of complex environments.

Summary updated

Abstract

Texture recognition is a challenging visual task as multiple perceptual attributes may be perceived from the same texture image when combined with different spatial context. Some recent works building upon Convolutional Neural Network (CNN) incorporate feature encoding with orderless aggregating to provide invariance to spatial layouts. However, these existing methods ignore visual texture attributes, which are important cues for describing the real-world texture images, resulting in incomplete description and inaccurate recognition. To address this problem, we propose a novel deep Multiple-Attribute-Perceived Network (MAP-Net) by progressively learning visual texture attributes in a mutually reinforced manner. Specifically, a multi-branch network architecture is devised, in which cascaded global contexts are learned by introducing similarity constraint at each branch, and leveraged as guidance of spatial feature encoding at next branch through an attribute transfer scheme. To enhance the modeling capability of spatial transformation, a deformable pooling strategy is introduced to augment the spatial sampling with adaptive offsets to the global context, leading to perceive new visual attributes. An attribute fusion module is then introduced to jointly utilize the perceived visual attributes and the abstracted semantic concepts at each branch. Experimental results on the five most challenging texture recognition datasets have demonstrated the superiority of the proposed model against the state-of-the-arts.
BibTeX
@inproceedings{zhai2019deep,
  author = {Wei Zhai and Yang Cao and Jing Zhang and Zheng-Jun Zha},
  title = {{Deep Multiple-Attribute-Perceived Network for Real-World Texture Recognition}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00371},
}
Figure