Abstract
In this paper, we propose a novel data augmentation strategy named Cut-Thumbnail, that aims to improve the shape bias of the network. We reduce an image to a certain size and replace the random region of the original image with the reduced image. The generated image not only retains most of the original image information but also has global information in the reduced image. We call the reduced image as thumbnail. Furthermore, we find that the idea of thumbnail can be perfectly integrated with Mixed Sample Data Augmentation, so we put one image's thumbnail on another image while the ground truth labels are also mixed, making great achievements on various computer vision tasks. Extensive experiments show that Cut-Thumbnail works better than state-of-the-art augmentation strategies across classification, fine-grained image classification, and object detection. On ImageNet classification, ResNet-50 architecture with our method achieves 79.21% accuracy, which is more than 2.8% improvement on the baseline.
Introduction
Despite their success, Convolutional Neural Networks (CNNs) suffer from a well-documented "texture bias." They tend to categorize objects based on local patterns, such as the texture of fur or feathers, rather than global shapes. For example, a CNN might classify a cat-shaped silhouette filled with elephant skin texture as an "elephant."
Existing augmentation methods like Cutout (deleting a patch) or CutMix (pasting a random patch from another image) help with regularization but do not explicitly encourage the network to learn shape. In fact, CutMix often pastes uninformative patches (like a piece of background), which can confuse the learning process. The authors sought a way to provide the network with complete semantic "shape" information in a compact, regularized format.
Methodology
The core idea is the Thumbnail: reducing an image to a fraction of its size (e.g., from 224x224 to 112x112). While local details are lost, the global shape remains recognizable.
The paper introduces three primary strategies:
- Self Thumbnail (ST): A thumbnail of Image A is pasted onto a random location within Image A. This forces the network to recognize the same object at multiple scales simultaneously.
- Mixed Single Thumbnail (MST): A thumbnail of Image B is pasted onto Image A. Labels are mixed based on a weighted average. Unlike CutMix, the thumbnail ensures that the "pasted" information contains the entire object from the second image.
- Mixed Multiple Thumbnails (MMT): Multiple thumbnails from different images are pasted onto a single background, increasing the "information density" of every training batch.
Comparison with Existing Methods
Cut-Thumbnail differs from traditional methods by ensuring that the augmented region contains global semantic information rather than random noise or partial textures.
As shown above, while Cutout removes information and CutMix introduces potentially irrelevant local patches, Cut-Thumbnail (e) preserves the global structure of the secondary object within the primary image's context.
Experimental Results
The authors evaluated Cut-Thumbnail across standard benchmarks including ImageNet-1K, CIFAR, and CUB-200-2011. Key findings include:
- ImageNet: A ResNet-50 trained with MST achieved 79.21% top-1 accuracy, a +2.89% improvement over the baseline and significantly higher than CutMix (78.40%).
- Fine-Grained Classification: On the CUB-200 bird dataset, Mixed Double Thumbnails (MDT) reached 86.72% accuracy.
- Object Detection: Applying Cut-Thumbnail as a pre-training strategy improved the mAP of RetinaNet on Pascal VOC from 70.14% to 72.16%.
- Shape Hypothesis: To prove the "shape" hypothesis, models were tested on grayscale versions of ImageNet. MST outperformed CutMix by a significant margin (+3.93% vs +2.91% over baseline), confirming it enhances shape-based learning.
Conclusion
Cut-Thumbnail is a simple yet powerful strategy that forces CNNs to overcome texture bias by preserving global structure while discarding local texture. While the method shows high sensitivity to hyperparameters like participation rates, its ability to increase information density makes it a superior alternative to random-patch methods like CutMix. Future work will explore dynamic label weighting and applications in semantic segmentation.
Summary updated