Abstract
While occluding parts of an image (e.g., "Cutout") helps models generalize on small datasets, it historically struggled to improve performance on large-scale benchmarks like ImageNet. This paper identifies joint training (batch augmentation) as the critical missing component. By training on both original and occluded versions of an image simultaneously, high-capacity models like ResNet50 achieve significant accuracy gains.
Introduction: The Problem of "Lazy" Networks
Deep neural networks often suffer from "photographer bias," where the subject of a photo is usually centered and clearly visible. Consequently, models tend to over-rely on a few "easy-to-recognize" features—such as a cat’s face or a specific texture—rather than understanding the object holistically. This makes them fragile; if those specific primary features are hidden, the model fails.
Researchers have attempted to address this using occlusions (blocking out parts of the image) to force networks to attend to secondary cues (like the cat's body). However, while this improved object localization, it often failed to boost, or even degraded, classification accuracy on large-scale datasets like ImageNet. This paper investigates this discrepancy and proposes a robust training strategy to overcome it.
Methods of Occlusion
The authors explore two primary categories of occlusion generation:
1. Stochastic Occlusions
- Hide-and-Seek (H&S): This method divides the image into a grid and randomly hides patches. It forces the model to find the object even when significant portions are missing.
- Cutout (CO): This simpler approach involves placing one or more random square "holes" in the image.
2. Saliency-based Occlusions
Instead of random masking, the authors utilize gradient-based saliency (specifically "NormGrad") to identify the most discriminative pixels for the model. By intentionally blocking these "important" areas, the model is forced to learn from less obvious but still relevant features. This targets the model's most discriminative features directly.
The Key Innovation: Joint Training
The authors argue that previous failures on ImageNet were due to a "distribution shift"—training only on occluded images makes the training data too different from the clean test set. They propose batch augmentation: for every batch of images, the model sees both the original images and an occluded version of those same images. This joint training allows the model to maintain its baseline performance while learning the robust features encouraged by occlusion.
Key Insights and Results
- Necessity of Joint Training: Seeing the unoccluded image in the same batch is essential for ImageNet performance. Without it, accuracy often drops.
- Architecture Capacity: High-capacity models like ResNet50 benefit significantly from large occlusions, whereas lower-capacity models like AlexNet show negligible gains. This suggests occlusion can serve as a measure of an architecture's robustness potential.
- Efficiency of Saliency: Saliency-based occlusion is highly efficient; hiding only 6% of an image via saliency can match the performance gains of hiding 50% randomly.
- Performance Gains: On ImageNet, ResNet50 with Cutout and joint training improved Top-1 accuracy by +0.85% (reaching 77.25%). Hide-and-Seek with joint training provided a +0.62% boost.
Conclusion
Occlusion is a powerful tool for regularization, provided it is paired with joint training. By forcing high-capacity models to look beyond the most obvious features without losing sight of the original data distribution, researchers can significantly improve the robustness and accuracy of large-scale image classifiers.
Summary updated