Convolutional networks see only a small neighborhood per layer. One cheap fix is the wavelet convolution: split the feature map into coarse and fine versions of itself using a wavelet transform, a fixed pair of averaging and differencing filters applied repeatedly at halving resolution. A small kernel is applied at every level and the results are reassembled. A 5×5 kernel at level $L$ then covers roughly $2^L \times 5$ pixels of the original image, while the parameter count grows only linearly in $L$. Wavelet Convolutions for Large Receptive Fields (WTConv) showed this works: it lifts ConvNeXt-T on ImageNet-1K from 81.0% to 81.7% top-1 and increases shape bias. It does so using only the Haar wavelet, the simplest one. Haar averages non-overlapping 2×2 blocks, and when the coarse levels are reassembled it copies each coarse value back into a block. The large kernel the layer actually applies is therefore piecewise constant on a dyadic grid, and it changes when the input shifts by one pixel.
This note argues that the choice of wavelet in these layers is not a matter of representational taste. It decides how blocky and how shift-dependent the enlarged receptive field is. It also argues that the hardware obstacle to smoother wavelets can be removed by writing them in "lifting" form, which factors any wavelet filter bank into short, in-place predict-and-update steps that resemble Haar's two-tap butterfly. The deliverable is a fused wavelet convolution for the CDF 5/3 and 9/7 biorthogonal wavelets, plus a controlled study of whether smoother bases improve dense prediction and shift stability at close to Haar's cost.
Core Claim
A WTConv layer computes a linear map: analysis filters, then a depthwise kernel on each subband, then synthesis filters. Because every level downsamples by two, this composite operator is not a single convolution. It is a family of $4^L$ different effective kernels, one for each position of a pixel within a $2^L \times 2^L$ cell. The mother wavelet sets two properties of this family. The synthesis low-pass filter sets its shape: Haar's box filter yields piecewise-constant kernels, while CDF 5/3's hat filter yields piecewise-linear ones. The overlap of the analysis filters sets how much the $4^L$ kernels differ from one another, which is a direct measure of aliasing. We claim that Haar maximizes this spread among common short wavelets. We further claim that reducing it with a smoother biorthogonal basis should matter most for tasks where WTConv's gains were smallest and alignment matters most, namely ADE20K segmentation (+0.6 mIoU in the WTConv paper) and COCO detection.
To make the spread measurable, define the shift-variance index of a layer as follows. Let $K_p$ be the effective kernel at cell phase $p$, and let $\bar K$ be the mean of $K_p$ over all $4^L$ phases. Then
$$ \mathrm{SVI} = \frac{\tfrac{1}{4^L}\sum_{p} \lVert K_p - \bar K \rVert_2^2}{\lVert \bar K \rVert_2^2}. $$
Each $K_p$ is obtained exactly by feeding impulses at every phase through the trained layer. An SVI of 0 means the layer behaves as an ordinary convolution with a very large kernel.
The engineering half of the claim answers the objection in Fast and Memory-Efficient Wavelet Convolutions via I/O-Aware Reformulation. That paper shows WTConv is limited by memory bandwidth rather than arithmetic, and it fixes this for Haar by fusing the butterfly and collapsing multi-level synthesis into one bit-indexed formula. Its authors say this does not extend to longer filters, because overlapping supports break the bit-indexed synthesis. We accept that and do not try to collapse synthesis. Lifting provides a different route. CDF 5/3 in one dimension consists of one predict step, $d[n] = x[2n+1] - \tfrac12(x[2n] + x[2n+2])$, and one update step, $s[n] = x[2n] + \tfrac14(d[n-1] + d[n])$. Each step is a three-tap in-place stencil whose exact inverse is obtained by flipping signs. A GPU tile can therefore run analysis, the per-subband kernels and inverse lifting entirely in on-chip memory. The only extra requirement is a halo, a border of neighboring pixels copied into each tile, that grows by a few pixels per level. The butterfly-fusion and scale-folding ideas of the Fast WTConv paper carry over. Only its collapsed synthesis is replaced.
Research Program
The program has three parts.
The first is analytical. We compute the effective kernels and SVI of Haar, CDF 5/3 and CDF 9/7 WTConv layers in closed form for random and trained kernels at $L = 1$ to $5$. We also relate SVI to the vanishing moments of the analysis high-pass filter, that is, the number of polynomial degrees the filter annihilates. Haar has one vanishing moment and CDF 5/3 has two. Generalized biorthogonal Daubechies wavelets gives the family of symmetric, compactly supported biorthogonal filters we draw from. Symmetry matters here because it avoids the phase shift that non-symmetric Daubechies filters would introduce into the effective kernel.
The second part is the fused lifting kernel, written in Triton or CUDA. It handles forward and backward passes and a halo scheme for tiling, and is benchmarked on the same sweep of channels $C \in \{32, 64, 128\}$, resolutions 128 to 512, and $L = 1$ to $5$ used by the Fast WTConv paper.
The third part is training and evaluation. ConvNeXt-T backbones use WTConv with each basis, run under the ImageNet-1K recipe of the original WTConv paper. They are then transferred to UperNet on ADE20K and Cascade Mask R-CNN on COCO, and tested on ImageNet-C and on one-pixel-shift consistency.
We also include a learnable variant, but deliberately as a test rather than the headline. Its lifting coefficients are initialized at CDF 5/3 and learned per stage, constrained so the predict weights sum to one. Lifting guarantees perfect reconstruction for any coefficients, so no soft penalty is needed. By contrast, Adaptive wavelet pooling for convolutional neural networks enforces reconstruction and alias cancellation through loss terms, which permit drift. Biorthogonal Tunable Wavelet Unit with Lifting Scheme in Convolutional Neural Network and Deep Adaptive Wavelet Network show that learned lifting helps when it replaces pooling in small networks, for example raising DTD accuracy from 33.99% to 43.72%. Both use lifting only for local 2× downsampling or for classification feature extraction. Neither makes it the engine of a multi-level receptive field, and neither measures what it does to the effective kernel.
Baselines
The naive reference point is ConvNeXt-T with its standard 7×7 depthwise convolution. It is reported at 81.0% ImageNet-1K top-1 in the WTConv paper and at 50.4 box AP with Cascade Mask R-CNN on COCO in ParCNetV2: Oversized Kernel with Enhanced Attention.
The strongest existing method on this exact question is Haar WTConv on ConvNeXt-T. It reports 81.7% top-1, a gain of 0.6 mIoU on ADE20K with UperNet (no absolute mIoU value found on Lacuna), an ImageNet-C mean corruption error reduction of about 1.2 points, and consistent but unquantified COCO gains (no reported box AP found on Lacuna for WTConv). Its fused implementation is reported at 1.27 to 1.50 times faster than a 7×7 depthwise convolution.
We will retrain Haar WTConv ourselves with the identical recipe, seeds, epochs and augmentation used for every lifting variant. Each basis gets the same hyperparameter budget, and each configuration is run with three seeds so that differences of a few tenths of a point can be separated from noise. For each basis we match $L$ and the kernel size, which keeps the nominal receptive field fixed and isolates the basis.
Beating Haar means two things. On quality, the lifting basis must gain at least 0.5 ADE20K mIoU, or produce a clearly better shift-consistency score, beyond seed noise, with ImageNet top-1 no lower. On cost, the fused lifting kernel must stay within 1.3 times the latency of fused Haar and no slower than a 7×7 depthwise convolution.
Two ablations isolate the key ingredient. The first uses the unfused CDF 5/3 layer, which separates the representational effect from the kernel engineering. The second is a "blurred Haar" layer, described in the next section, which separates smoother analysis from smoother synthesis. For context only, we compare against ParCNetV2 (84.6% top-1 at the B size, 52.6 box AP at the T size) and CCNet (45.76% ADE20K mIoU). Their model scales differ, so they are not treated as matched competitors.
Why not just…?
Why not just swap Haar for a fixed smoother wavelet from an off-the-shelf wavelet library and skip the new kernel? This is the most serious threat, and the proposal is built around it rather than against it. If fixed CDF 5/3 already captures the gains, then the contribution is exactly the fixed-basis study plus the kernel that makes it affordable. Without that kernel, the Fast WTConv paper's own analysis implies the unfused smooth version would bring back the memory-bandwidth slowdown it removed. The unfused CDF 5/3 arm measures the quality effect, and the fused arm measures whether it survives at practical speed. We do not claim a learned basis beats a good fixed one. That comparison is an open question the experiments answer.
Why not just blur before the Haar downsampling, as in standard anti-aliasing? Blurring smooths the analysis side only. It breaks perfect reconstruction and leaves Haar's box-shaped synthesis, so the effective kernels stay piecewise constant. The blurred-Haar ablation tests directly whether analysis smoothing alone lowers SVI and recovers the dense-prediction gain. If it does, our argument that the basis must be changed as a whole is weakened.
Why not just use an undecimated wavelet or dilated convolutions, which have no shift variance at all? Both remove aliasing by keeping every level at full resolution. That multiplies activation memory by the number of levels and gives up the exponential cheapness that makes WTConv attractive. We include a dilated depthwise stack with a matched receptive field as a baseline, and report its latency and memory alongside its accuracy.
Why not just use large or oversized kernels? ParCNetV2 reports strong numbers with global one-dimensional kernels. The WTConv paper reports that brute-force large kernels grow parameters quadratically and saturate in optimization. These alternatives answer a different question, namely whether to use wavelets at all, and they are included only as context.
Proposed Experiments
The first decisive check is mechanistic. Before any large training run, we measure SVI and effective-kernel smoothness for Haar, CDF 5/3 and CDF 9/7 layers from trained ImageNet checkpoints, at every stage of the network. The prediction is Haar > 5/3 > 9/7 in SVI.
The second check is predictive. Across bases and the blurred-Haar ablation, SVI should rank one-pixel-shift prediction consistency on ImageNet and boundary IoU on ADE20K. If SVI differs across bases but downstream metrics do not, then the network is compensating for its basis, and the representational claim fails.
The third check is downstream. It compares ADE20K mIoU, COCO box and mask AP, ImageNet-C mean corruption error and shape bias against the retrained Haar WTConv. We expect the largest gains in segmentation and the smallest in ImageNet top-1, because classification tolerates misalignment.
The fourth check is on cost. Latency and peak memory of the fused lifting kernel are measured across the Fast WTConv benchmark sweep. The project counts as successful only if the quality gain holds at or below the cost of a 7×7 depthwise convolution.
The fifth check concerns the learnable lifting variant. We report whether it moves away from its CDF 5/3 starting point. Following the finding in Wavelet-Based Mechanistic Interpretability of Vision Transformers via Frequency-Aware Ablations that ViT representations depend strongly on edge information captured by Haar (a Haar high-frequency ablation MSE of 0.027 versus 0.015 for db2), we also report whether early stages drift toward sharper, Haar-like filters while later stages stay smooth. Such a split would argue for choosing the basis per stage rather than using one wavelet throughout.
References
- Wavelet Convolutions for Large Receptive Fields
- Fast and Memory-Efficient Wavelet Convolutions via I/O-Aware Reformulation
- Generalized biorthogonal Daubechies wavelets
- Adaptive wavelet pooling for convolutional neural networks
- Biorthogonal Tunable Wavelet Unit with Lifting Scheme in Convolutional Neural Network
- Deep Adaptive Wavelet Network
- ParCNetV2: Oversized Kernel with Enhanced Attention
- Wavelet-Based Mechanistic Interpretability of Vision Transformers via Frequency-Aware Ablations