Truncated Filter Bases as a Cheap Route to Alias-Free Convnets

Convolutional networks change their predictions when an image moves by a single pixel. Tested models flip their top prediction up to 30% of the time under a 1-pixel downward shift (Azulay et al.). The best-understood cure is aliasing-free design. Aliasing is the effect where fine detail, sampled too coarsely, folds back and shows up as spurious coarse patterns. The Alias-Free Convnet (Michaeli et al.) removes it everywhere by doing three things: it swaps ReLU for low-degree polynomials, it wraps every activation in an upsample, low-pass, downsample sandwich, and it uses circular padding. It reaches 100% shift consistency on ImageNet, but training is 5× slower and inference 6.5× slower. A separate line of work restricts convolution filters to a few smooth, low-frequency basis functions (DCFNet). That makes networks smaller and more stable to deformation, but it has never been used to control aliasing.

We propose that these two ideas fit together exactly. Suppose every filter is restricted to frequencies below $\pi/d$, and each activation is a degree-$d$ polynomial. Then the signal after the activation is still below the sampling limit at the current resolution. The expensive upsample-and-filter wrapper becomes unnecessary, because the filters themselves supply the anti-aliasing and the next layer's filters restore the band limit. Put in two sentences: basis truncation of the weights is a way to pay for alias-free activations in advance. It moves the cost of shift invariance from a slow per-activation filtering step into a cheaper constraint on which filters the network may learn.

Core Claim

Take a feature map sampled at a given rate, where $\pi$ denotes its highest representable frequency (the Nyquist limit). Two facts about frequency content drive the design. Convolving with a filter whose spectrum is supported on $|\omega| \le \pi/d$ gives an output supported there too. A degree-$d$ polynomial of a signal band-limited to $B$ is band-limited to $dB$, because products of signals add their frequencies; this is the exact bandwidth expansion AFC relies on. Chaining these gives a closed cycle of three steps. First, a truncated filter restricts content to $\pi/d$. Second, the polynomial expands it to at most $\pi$ with no folding. Third, the next truncated filter restricts it back to $\pi/d$. When that filter is strided by 2 and $d = 2$, the content is already below the new Nyquist limit $\pi/2$ before subsampling. No layer ever needs to change sampling rate except to downsample.

Filters with a $k \times k$ spatial footprint cannot be exactly band-limited. The real quantity to control is therefore each layer's out-of-band energy fraction $\varepsilon_\ell$, the share of filter energy above $\pi/d$. We expect the aliased energy injected at layer $\ell$ to be bounded, to first order, by a constant times $\varepsilon_\ell$ times the input energy. Shift inconsistency should then grow with $\sum_\ell \varepsilon_\ell$. This mirrors the depth-wise accumulation of equivariance error that RST-CNN identifies for discretized basis filters. The claim is testable: consistency should degrade in a predictable way as the truncation is loosened.

Relation to Prior Work

The weight-side work supplies the method. DCFNet writes each filter as $W = \sum_{k=1}^{K} a_k \psi_k$, with fixed Fourier-Bessel bases $\psi_k$ and learned coefficients $a_k$. It computes the convolution as a shared basis-filtering step followed by a $1\times1$ mixing step, and matches full VGG-16 on face verification with about a third of the parameters and GFLOPs. ScDCFNet and RST-CNN extend the same truncation to scale and rotation groups and prove deformation-stability bounds. All three keep ordinary ReLU, strided convolution and pooling. RST-CNN's authors note that sampling breaks their continuous guarantees, and none of the three measure pixel-shift consistency. We reuse their basis decomposition but choose the truncation for a different purpose: meeting a sampling condition rather than resisting deformation.

The anti-aliasing work motivates the proposal and constrains its design. BlurPool inserts a fixed blur before subsampling and gains +0.7% to +0.9% top-1 on ResNet-50 with 1.3% to 2.1% better consistency. AFC shows why BlurPool cannot finish the job: ReLU injects unbounded frequencies before any blur is applied. This negative result forces polynomial activations into our design. AFC keeps unconstrained pixel-grid filters and buys band-limitedness with multi-rate filtering. We instead buy it with filter truncation, and its reported cost is the target we aim to cut.

Two further methods reach high consistency with other machinery. Gaussian-Hermite Sampling projects patches onto a truncated orthogonal basis at pooling time and reaches 100% consistency on CIFAR-10. It is about 10× slower than max-pooling and, by its authors' account, unsuited to dense prediction because it anchors on a max pivot. TIPS reaches 92.87% ImageNet consistency with learned polyphase mixing. Both act only at downsampling layers. Ours acts on the weights and leaves the architecture's feature maps equivariant, so it transfers to detection.

The boundary work constrains how the claims can be scored. Kayhan and van Gemert, Mind the Pad and Padding Aware Neurons show that zero padding lets filters locate the image border. AFC's certified invariance holds only for circular (wrap-around) shifts. We therefore report both circular shifts and crop-shifts, meaning translations in which new content enters the frame. We treat the second as the honest test, where no method, ours included, is certified.

Research Program

We will build an alias-free ConvNeXt variant starting from the official AFC code (hmichaeli/alias_free_convnets). We keep its polynomial activations and its alias-free normalization, and delete the upsample-activate-downsample wrapper. Every depthwise and strided convolution is re-parameterized in a truncated basis.

We will compare two bases. The first is DCFNet's Fourier-Bessel basis. The second is a basis chosen explicitly to minimize out-of-band energy on the $k\times k$ support: the leading eigenvectors of the operator that band-limits to $\pi/2$ and then restricts to the support, the discrete analogue of prolate spheroidal functions. Fourier-Bessel bases are smooth but not designed for spectral concentration, so this comparison tests whether the choice of basis matters or only the fact of truncation.

We will measure $\varepsilon_\ell$ per layer before and after training. The coefficients $a_k$ can recombine basis functions, but they cannot leave the basis span. This means $\varepsilon_\ell$ is bounded by the worst basis function's leakage, and the bound can be checked analytically.

The mixing step is a $1\times1$ convolution and the basis step is a small fixed depthwise convolution. Both run on standard dense kernels, so wall-clock comparisons need no custom CUDA work. The DCFNet line lists custom kernels as an open problem, and this construction sidesteps it by putting the fixed basis in the depthwise path, where ConvNeXt already spends little compute.

A secondary analysis asks the open question from the padding literature: can filters confined to smooth low-frequency modes still become padding-aware neurons? We expect they can, since a smooth filter still responds strongly to the step at a zero-padded border. We will count such neurons using the criterion of Padding Aware Neurons under zero, reflect and circular padding. The result decides which padding the crop-shift experiments should use; it is not an argument for the method.

Baselines

The naive reference point is a standard ConvNeXt-Tiny trained with the AFC repository's recipe and ordinary random-crop augmentation. Its reported ImageNet top-1 is 82.1%, dropping to 73.6% under AFC's black-box translation attack (AFC). It fixes the accuracy ceiling and the speed floor.

The strongest existing method is AFC itself. It is the only CNN in the file with 100% integer and fractional shift consistency on ImageNet, at 81.04% top-1 retained under translation attack, and at 5× training and 6.5× inference cost (AFC). We will use the released checkpoints and also retrain it under our exact schedule and hardware so that timings are measured, not quoted. Beating AFC does not mean higher accuracy. It means staying within one point of AFC's top-1 and at or above 99% consistency on circular fractional shifts, with training and inference throughput at least 2.5× faster than AFC on identical GPUs. A method that only matches AFC's speed gives no reason to adopt it.

BlurPool ConvNeXt, with a blur before each subsampling and ReLU/GELU activations kept, represents the cheap drop-in route. We will train it with the same schedule. It frames how much of the gain simply comes from anti-aliasing the downsampling layers.

The two ablations isolate the key ingredient. The first removes truncation: AFC's polynomial activations without the wrapper, on unconstrained filters. This network should alias badly, which shows that the wrapper, or something replacing it, is necessary. The second, blur-before-activation, keeps unconstrained filters and applies a fixed low-pass blur to the pre-activation map in place of the wrapper. This is the ablation that decides whether learned truncation adds anything beyond cheap fixed filtering. All variants share data, epochs, augmentation, optimizer and a fixed learning-rate sweep of the same size, with truncation level $K$ counted as part of our tuning budget.

For the shift-consistency ranking on non-alias-free designs, TIPS reports 92.87% ImageNet consistency against 89.05% for max-pooling (TIPS). On CIFAR-10, GHS reports 95.54% accuracy with ResNet-18 and 100% consistency (GHS). We will quote these rather than rerun them.

Why not just…?

Why not just use BlurPool? Because BlurPool filters after the activation has already created frequencies that fold back. AFC shows it cannot reach full consistency. In detection, Manfredi and Wang report that it cuts shift sensitivity by only about 30%. Our BlurPool baseline will show the size of that gap under matched training.

Why not just blur before each activation instead of learning truncated filters? This is the real threat, since a fixed blur composed with any filter is itself a low-pass filter. If it matches our method on consistency and accuracy, the contribution shrinks to what the basis decomposition buys in parameters and speed. We will report exactly that outcome rather than argue around it. Our reasons to expect a difference are two. The fixed blur adds a convolution per activation that truncation folds into the existing filter. Blurs short enough to be cheap, such as the binomial kernels used in BlurPool, leak substantially above $\pi/2$, whereas a concentration-optimized basis minimizes that leakage for the same footprint. The $\varepsilon_\ell$ measurements and the consistency curves will settle it.

Why not just train with random shifts? Azulay et al. find that augmentation widens stability only around typical framings and leaves 1-pixel prediction flips common. Manfredi and Wang find augmented detectors still swing by up to 21.5 AP points under 15-pixel shifts. The naive baseline already includes standard augmentation.

Why not just speed up AFC's own filtering? Faster FFT implementations would narrow the speed gap without any change in design. We therefore compare against AFC timed on the same hardware and report FLOPs alongside wall clock. Our method removes a whole resolution-doubling step per activation, which is a FLOP reduction that no implementation of AFC can recover.

Proposed Experiments and Decisive Checks

On CIFAR-10, a fast grid over truncation level $K$, polynomial degree and basis type will trace the curve of consistency against $\sum_\ell \varepsilon_\ell$. The core claim predicts a monotone relation, with consistency near 100% on circular shifts once leakage is small. Shift consistency here follows the standard definition: agreement of top-1 predictions between two random integer or fractional shifts of the same image. If this curve is flat or noisy, the band-limit mechanism is not what drives invariance, and the proposal fails at its foundation.

On ImageNet-1K with ConvNeXt-Tiny, we report top-1 accuracy, integer and fractional circular-shift consistency, crop-shift consistency, AFC's translation-attack accuracy, and measured training and inference throughput against every baseline above. The decisive check is the accuracy cost of the bandwidth restriction. Features carrying only half the representable band may lose fine detail, a limitation the DCFNet line itself acknowledges. If top-1 falls more than about a point below AFC, we will test whether a modest input-resolution increase recovers it while still beating AFC's cost.

Finally, we move the backbone into RetinaNet on MS-COCO and measure the variation in AP at IoU 0.5 under 1-pixel shifts, following Manfredi and Wang. They report a swing of 5.1 points for standard RetinaNet under this protocol. For the baselines here, no reported result was found on Lacuna for an AFC backbone in detection. Detection tests the advantage we claim over GHS and TIPS: an alias-free backbone that stays equivariant and is cheap enough to use for dense prediction.