Abstract
Deep Convolutional Neural Networks (DCNNs) utilize down-sampling layers, such as pooling or strided convolutions, to reduce spatial dimensions and achieve translation invariance. However, the standard fixed-stride approach often results in significant information loss because kernels only sample from a rigid grid. This paper introduces Random Shifting, a lightweight training strategy that randomly offsets kernel centers during training. By jittering the receptive field, Random Shifting acts as a form of internal data augmentation, forcing the network to learn more robust features. Experimental results on ImageNet and CIFAR-10 demonstrate improved accuracy with virtually no additional computational cost.
Introduction: The Rigidity of Fixed-Stride Sampling
In traditional DCNN architectures, down-sampling is critical for controlling parameter counts and expanding receptive fields. Despite their utility, these layers typically follow a fixed stride strategy. For instance, a $2 \times 2$ pooling kernel with a stride of 2 only ever processes pixels at specific, predetermined coordinates.
This rigidity creates a bottleneck: if a salient feature shifts by a single pixel, it may fall into a different pooling window or be discarded entirely. This sensitivity to small translations limits the network's ability to generalize. The authors argue that this "fixed grid" approach is a primary source of information loss in deep networks.
Proposed Method: Random Shifting
To address this, the authors propose Random Shifting. Instead of always starting a down-sampling kernel at the top-left corner (offset 0,0), a random offset $O$ is added to the kernel center during each training iteration.
Implementation Details
The process involves three main steps:
- Constraint Calculation: To keep the kernel within the feature map boundaries, the maximum shift $r$ is defined by the stride ($s$), kernel size ($k$), and padding ($p$): $r = \min(s - 1, (k - 1)/2 - p)$.
- Randomization: At each training step, an offset $O$ is randomly sampled from the domain $\{-r, \dots, 0, \dots, r\}$.
- Integration: This strategy is compatible with any down-sampling layer, including max-pooling and strided convolutions.
Furthermore, the authors suggest combining this with Fully Convolutional Network (FCN) style training. In this mode, the network produces a grid of outputs, allowing it to learn from multiple shifted versions of the same input simultaneously, which enhances training efficiency.
Experimental Evaluation
The researchers evaluated Random Shifting across several benchmarks, including CIFAR-10, CIFAR-100, ImageNet (2012), and the SIFT Flow dataset for semantic segmentation.
Key Results on ImageNet
On the large-scale ImageNet dataset using AlexNet as a baseline, the inclusion of Random Shifting in deeper layers (specifically pool2 and pool5) provided a measurable boost in performance.
As shown in the table above, the combination of SPool2 and SPool5 (Shifting Pooling layers) improved top-5 accuracy from a baseline of 80.19% to 80.80%.
Performance on CIFAR and Segmentation
- CIFAR-10: Combining Random Shifting with standard horizontal flipping achieved an accuracy of 91.72%, outperforming both the baseline and traditional cropping methods.
- Semantic Segmentation: On the SIFT Flow dataset, integrating Random Shifting into an FCN-16s model improved Mean Accuracy from 51.7% to 52.4%.
Insights and Discussion
Random Shifting functions as a form of "deep" data augmentation. Unlike standard augmentation that occurs at the input layer, Random Shifting forces invariance at every down-sampling stage. This allows the network to "mine" information that would otherwise be lost to a fixed grid.
A key finding was the importance of strategic placement. For smaller images (CIFAR), shifting in lower layers can introduce instability. Conversely, for high-resolution inputs (ImageNet), shifting deeper layers like pool5 is highly effective because the relative change in the receptive field is more significant for the final classification.
Conclusion and Limitations
Random Shifting offers a "free" performance boost due to its negligible computational cost. However, it does introduce some sensitivity to hyperparameters; the authors noted that the initial learning rate often requires downward adjustment to ensure convergence. Future research could explore non-linear offset patterns and the interaction of this strategy with modern architectures like ResNets or Transformers.
Summary updated