Abstract
Standard Convolutional Neural Networks (CNNs) are known to be "texture-biased," often relying on local patterns rather than global shapes for object recognition. While increasing "shape-bias" has been proposed as a universal solution for out-of-distribution (OOD) robustness, this paper demonstrates that the optimal bias is category-dependent. We introduce Shape-Texture Adaptive Recombination (STAR), a framework that dynamically weights shape and texture features on a per-instance basis, achieving state-of-the-art robustness across multiple benchmarks.
The Problem: Is Shape-Bias a Universal Cure?
Humans are predominantly shape-biased, leading researchers to hypothesize that forcing CNNs to prioritize shape would bridge the gap in OOD robustness—improving performance on blurred, distorted, or artistic images. However, this assumption overlooks two critical factors:
- Category Sensitivity: Shape is not always the most discriminative feature. For many animal species, texture (e.g., fur patterns, stripes) is more informative than a generic silhouette.
- Scenario Sensitivity: Under certain corruptions like heavy blur, shape information becomes unreliable, making texture a more stable cue.
Proposed Method: STAR
To address these limitations, we propose Shape-Texture Adaptive Recombination (STAR). Rather than enforcing a static bias, STAR learns to adaptively combine features based on the input image.
The architecture consists of:
- Specialized Heads: A shared backbone branches into three heads: a Shape-biased head ($H_s$) trained on texture-randomized images, a Texture-biased head ($H_t$) trained on shuffled image patches, and a Debiased head ($H_d$) trained with standard augmentations.
- Recombination Head ($H_r$): This module predicts instance-specific weights ($\alpha, \beta, \gamma$) to fuse the outputs of the specialized heads. For instance, if an image is blurred, $H_r$ shifts weight toward the texture head.
The Animal/Artifact Divide
A key finding of this work is the divergence in feature dependency between natural and man-made objects. As illustrated in Figure 1, texture is often sufficient to identify animals even when poses change, whereas shape is the primary identifier for artifacts.
Our analysis shows that for animals, texture-biased models can outperform shape-biased ones in OOD settings. Conversely, for artifacts, shape remains the dominant and most robust cue.
Experimental Results
We evaluated STAR on a newly curated Category-Balanced ImageNet and several robustness benchmarks (ImageNet-C, ImageNet-R, Sketch, etc.).
- OOD Robustness: STAR achieved an average robustness of 57.2%, significantly outperforming vanilla ResNet-50 (42.7%) and standard shape-biasing methods like Stylized-ImageNet (50.7%).
- Adversarial Robustness: STAR showed remarkable resilience to FGSM attacks, scoring 59.3% compared to the baseline's 35.0%.
- Adaptive Weighting: Qualitative checks confirm that STAR intelligently adjusts its weights—favoring texture for blurred images and shape for pixelated ones.
Conclusion
The quest for robust computer vision should move beyond the "shape-bias is better" dogma. Robustness is not about a single universal bias but about the ability to select the most reliable cues for a given object in a given environment. STAR provides a flexible mechanism to achieve this, paving the way for more adaptable and resilient AI systems.
Summary updated