Windowed Band-Limiting for Crop-Shift Invariant Convnets

Image classifiers and detectors change their answers when the picture moves by a single pixel. A line of work has fixed this by making every layer "band-limited," meaning feature maps are kept free of the fine-grained, rapidly varying patterns that get scrambled when a network subsamples its features. The best of these methods reach perfect shift consistency, but only for circular shifts, where pixels pushed off one edge reappear on the opposite edge. Real camera motion is a crop shift: content leaves one side of the frame and new content enters the other. This note argues that the circular guarantee is an artifact of how these methods remove high frequencies. They project onto a periodic Fourier basis and cut it off. That operation treats the image border as a sharp jump, and its ringing spreads across the whole feature map. We propose replacing the periodic cutoff with a cutoff in a basis made for finite, non-periodic windows: the discrete prolate spheroidal (Slepian) sequences. These are the signals that pack the most energy into a fixed frequency band while living on a fixed finite interval.

In two sentences: periodic spectral truncation turns a border event into a global perturbation, and truncating in a window-adapted band-limited basis should confine that perturbation to a thin strip near the edges. If so, crop-shift consistency should approach circular-shift consistency. The operator is also a small fixed matrix multiply, so this should come at a fraction of the cost of current alias-free networks.

Core Claim

Our claim is about where the remaining crop-shift error comes from, and it can be tested directly. In alias-free convnets such as Alias-Free Convnets, low-pass filtering is done by taking a Fourier transform, discarding frequencies above a cutoff, and transforming back. Under a crop shift the content at the wrap-around seam changes. Because the ideal periodic cutoff has slowly decaying sinc-shaped tails, the change contaminates features far from the border. We predict that the feature difference between two crop-shifted inputs, aligned on their shared interior, decays slowly with distance from the border for periodic truncation and quickly for Slepian truncation. We further predict that this difference in decay accounts for most of the gap between circular and crop consistency.

The Slepian basis also gives a principled answer to a question the spectral-truncation literature leaves empirical: how many basis functions to keep. For a window of $N$ samples and a half-bandwidth $W$ (in cycles per sample), the Slepian sequences are the eigenvectors $s_k$ of the $N\times N$ matrix $B_{ij}=\sin(2\pi W(i-j))/(\pi(i-j))$. Their eigenvalues $\lambda_k$ measure how much of each vector's energy lies inside the band. Roughly $2NW$ eigenvalues are close to 1 and the rest fall to near 0, which is the classical time–bandwidth count. So the cutoff $K$ is set by the stride rather than tuned. Our downsampling operator along one axis is

$$y = E_s\, S_K S_K^{\top} x,$$

where $x$ is a feature row of length $N$, $S_K$ is the $N\times K$ matrix of leading Slepians, $K=\lceil 2NW\rceil$ with $W=1/(2s)$ for stride $s$, and $E_s$ keeps every $s$-th sample. The operator is applied separably along rows and columns. It is a fixed $N/s \times N$ matrix, so it costs two small matrix multiplies per feature map and needs no FFT.

Relation to Prior Work

Several papers motivate the problem. Why do deep convolutional networks generalize so poorly to small image transformations? and Making Convolutional Networks Shift-Invariant Again show that subsampling without low-pass filtering breaks shift invariance. The second paper repairs this partially with a fixed blur before striding ("BlurPool"). We keep its placement but change what the filter does at the borders. On Translation Invariance in CNNs and Mind the Pad show that zero-padding lets networks encode absolute position. That is the same border problem our operator targets, approached through padding rather than spectra.

The certified methods constrain our design. Alias-Free Convnets and Alias-Free ViT achieve exact fractional circular-shift invariance through FFT truncation, with polynomial activations in the convnet and linear attention in the transformer. Both acknowledge that crop shifts are not covered, and both pay a large compute cost (5× training and 6.5× inference slowdown for AFC; 487 vs. 69 training hours for AFT). We do not try to keep their exact circular guarantee. We trade it for an approximate guarantee that degrades only near the border under the realistic shift model.

Accurate Shift Invariant CNNs Using Gaussian-Hermite Moments supplies the closest method. It projects pooled patches onto Gaussian-windowed Hermite polynomials and anchors them at the activation peak. Our work differs in three ways. Its consistency guarantee is again circular. Its anchoring at the peak makes the output depend on a single maximum. It is reported to be about 10× slower than max-pooling. Slepians are the band-limited analogue of its Gaussian-windowed basis, but they are optimal for a hard window and need no factorials. DCFNet and RST-CNN truncate the filters in a Fourier-Bessel basis. RST-CNN proves that low-mode truncation bounds deformation error. We borrow the idea that truncation order controls stability, but we apply it to feature maps on a finite window, where the boundary matters, instead of to the filter weights.

Research Program

We first build the operator as a drop-in module with precomputed, cached $S_K$ for each feature-map size. Next we install it in two settings. In the first, it replaces BlurPool and strided subsampling in ResNet-18/50, with ReLU and zero-padding unchanged. In the second, it replaces every periodic FFT truncation inside an AFC-style ConvNeXt-Tiny, including the low-pass filters after polynomial activations. In that setting we also substitute "valid-window" convolution: the convolution output near the edges is computed by projecting onto the Slepian subspace of the true image support instead of inventing padded values.

The analysis has three parts. We measure the boundary-contamination profile layer by layer, defined as the interior-aligned feature difference against distance from the border. We sweep $K$ around $2NW$ to test whether the time–bandwidth count is the right cutoff, which would show up as a knee in the accuracy–consistency curve near it. We also derive a bound on interior equivariance error as a function of $\sum_{k\le K}(1-\lambda_k)$ and distance from the window edge, in the spirit of the RST-CNN truncation bound.

Baselines

The naive reference is a standard ResNet-50 or ConvNeXt-Tiny trained with the usual random-crop augmentation. This is what practitioners deploy, and it sets the floor for crop-shift consistency.

The cheap competitor is BlurPool from Making Convolutional Networks Shift-Invariant Again. It reports +1.3 to +2.1 points of ImageNet consistency and +0.7 to +0.9 top-1 on ResNet-50. It matters because it occupies exactly the slot our operator occupies. We run it with reflect padding as well as zero padding, so that it is not handicapped by a weak padding choice.

The strongest adaptive-sampling competitor is TIPS, from Translation Invariant Polyphase Sampling. It reports 92.87% ImageNet shift consistency against 89.05% for max-pooling.

The strongest certified competitor is AFC. It reports 100% circular consistency, 81.04% clean ImageNet top-1, and 81.04% under black-box translation attack, where baseline ConvNeXt falls from 82.1% to 73.6%. We found no reported crop-shift consistency number for AFC on Lacuna, so we will measure it ourselves with the authors' configuration.

All models share the same training recipe, epochs, augmentation, and hyperparameter search budget. We report wall-clock training and inference time on identical hardware.

The key ablation isolates the basis. It swaps the Slepian projection for a periodic FFT truncation with an identical cutoff and identical placement, and everything else stays fixed.

We would count the method as beating AFC if it achieves crop-shift consistency clearly above AFC's measured crop-shift value and clean accuracy within half a point. The run must also cost less than half of AFC's training time. It beats BlurPool and TIPS if it gives higher crop consistency at equal or better clean accuracy and comparable throughput.

Why not just…?

Why not just taper the image with a Hann window before the FFT cutoff? This is the most serious threat. Tapering is the textbook fix for spectral leakage, and Slepians are in effect the optimal taper. We include a Hann-tapered FFT variant at the same cutoff. If it matches Slepian truncation, the finding still stands: border handling, not band-limiting, is the bottleneck for crop shifts. The contribution would then shift to the diagnostic and the cheap tapered operator, and we will report it that way. We expect Slepians to win at high truncation ratios, where a fixed taper wastes the samples near the edges.

Why not just use reflect padding plus BlurPool and train with crop augmentation? The augmentation literature (Why do deep convolutional networks generalize so poorly…; Minimal Images) shows that training on shifts does not produce consistency on unseen shifts. BlurPool also leaves activation-induced aliasing in place (Alias-Free Convnets). Still, reflect padding plus BlurPool is cheap, so it runs as a full baseline, and the boundary-contamination profile will show whether it already localizes border errors.

Why not just evaluate AFC with circular padding and accept the gap? Circular padding joins unrelated content at the seam, and our hypothesis is precisely that this seam causes non-local error. The periodic-versus-Slepian ablation inside the AFC backbone tests this directly.

Proposed Experiments

On CIFAR-10 we train ResNet-18 with each downsampling variant and compare clean accuracy against the GHS paper's reported 95.54% (GHS), 94.48% (APS), and 91.96% (baseline). We measure consistency under circular shifts and under true crop shifts. Crop shifts are produced by cropping offset windows from a padded-then-upsampled source, and fractional offsets are made by band-limited resampling of a larger image before cropping.

On ImageNet-1K we train ResNet-50 and ConvNeXt-Tiny variants and report top-1, circular and crop consistency at integer and fractional offsets, accuracy under the AFC translation attack, and throughput.

On COCO we put the ResNet-50 backbones into RetinaNet and follow the protocol of Shift Equivariance in Object Detection. That paper reports $\Delta AP_{50}$ of 5.1 at a 1-pixel shift and 21.5 over 15-pixel shifts for RetinaNet, and only about a 30% reduction from BlurPool. Detection is where crop shifts dominate, because objects move relative to the frame edge.

Three results would be decisive. First, the contamination profile should decay over a few pixels for Slepian truncation and over the whole map for periodic truncation. Second, closing that difference should close most of the crop-versus-circular consistency gap. Third, the $K$ sweep should show a knee near $2NW$. If the first holds but the second does not, then border effects are not the main remaining cause of crop-shift error, and the proposal is refuted.