Abstract
This paper starts by revealing a surprising finding: without any learning, a randomly initialized CNN can localize objects surprisingly well. That is, a CNN has an inductive bias to naturally focus on objects, named as Tobias ("The object is at sight") in this paper. This empirical inductive bias is further analyzed and successfully applied to self-supervised learning (SSL). A CNN is encouraged to learn representations that focus on the foreground object, by transforming every image into various versions with different backgrounds, where the foreground and background separation is guided by Tobias. Experimental results show that the proposed Tobias significantly improves downstream tasks, especially for object detection. This paper also shows that Tobias has consistent improvements on training sets of different sizes, and is more resilient to changes in image augmentations.
Introduction
While Convolutional Neural Networks (CNNs) are the cornerstone of modern computer vision, the nature of their inductive bias—the innate assumptions a model makes to generalize—remains partially obscured. Standard object localization techniques typically rely on models pre-trained on massive labeled datasets like ImageNet. This research investigates a fundamental question: Is the ability to distinguish an object from its background a learned skill, or is it an inherent property of the CNN architecture itself?
The authors discover that CNNs possess a "born gift": even with completely random weights and zero training, they can localize objects with remarkable accuracy. This phenomenon, termed Tobias (The object is at sight), suggests that the architectural design of CNNs naturally favors object-centric features over background textures.
Methodology: Leveraging the Tobias Bias
The paper introduces a two-part framework to exploit this architectural bias for self-supervised learning (SSL).
1. Localization via Randomness
Using a randomly initialized ResNet-50, the authors apply Selective Convolutional Descriptor Aggregation (SCDA). By aggregating feature maps from the final convolutional layer and applying a threshold (typically the median activation), they generate a binary mask that separates the image into "foreground" and "background" regions.
2. The Tobias Augmentation
To improve SSL, the authors propose a background-swapping augmentation. An image is divided into a grid (e.g., 4x4). Using the mask from the random CNN, patches are identified as either foreground or background. During training, new "views" of an image are created by preserving the foreground patches of the target image while replacing the background patches with those sampled from a pool of other images.
This augmented data is integrated into standard contrastive learning frameworks like MoCo or SimCLR. By forcing the encoder to recognize the object's identity across varying backgrounds, the model learns to focus on the "essence" of the object rather than environmental clutter.
Analysis of the Inductive Bias
The authors conducted extensive ablations to understand why random CNNs exhibit this behavior.
- The Role of ReLU: The primary driver appears to be the ReLU activation function. Backgrounds are often smoother or less textured than objects. ReLU is more likely to deactivate (output zero) for these low-texture regions, effectively filtering them out. Replacing ReLU with Sigmoid or Arctan causes the localization ability to vanish.
- Depth and Architecture: Deeper random networks tend to localize better than shallow ones. As signals propagate through successive layers of random convolutions and ReLUs, the contrast between the "noisy" object and the "quiet" background is amplified. Interestingly, this bias is specific to CNNs; Vision Transformers (ViT) do not show similar innate localization when randomly initialized.
Experimental Results
The effectiveness of the Tobias bias was evaluated across several benchmarks, including ImageNet, CUB-200, and Pascal VOC.
- Localization Performance: A randomly initialized ResNet-50 achieved 48.2% localization accuracy on ImageNet, which is strikingly close to the 51.9% achieved by a fully supervised model.
- Downstream Task Improvement: In SSL settings, MoCov2-Tobias trained for 200 epochs outperformed the baseline MoCov2 trained for 800 epochs on object detection tasks. On Pascal VOC, the method consistently improved the $AP_{75}$ metric, indicating superior capture of object boundaries.
- Data Efficiency: On a small subset of 10,000 ImageNet images, Tobias improved linear evaluation accuracy on Tiny-ImageNet from near-random (0.5%) to 9.9%, demonstrating its value in low-data regimes.
Conclusion and Future Work
The discovery of the Tobias bias reveals that CNN architectures are fundamentally "pre-wired" to perceive objects. By leveraging this bias through background-swapping augmentations, self-supervised models can achieve faster convergence and better localization without human labels.
However, limitations remain. The current 4x4 grid approach is coarse, and the assumption that images are split 50/50 between foreground and background is a simplification. Future research may explore finer-grained augmentations and the application of this bias to more complex, non-convolutional architectures.
Summary updated