Restoring Shift Invariance with Anti-Aliased Downsampling

This direction is split into other directions: anti-aliased downsampling. Its last version stays readable and citable here.

Version 1 of 139 papers50 conceptsfrozen 2026-10-04

It is a common misconception that Convolutional Neural Networks (CNNs) are inherently translation-invariant. While convolutions themselves are equivariant—shifting the input shifts the feature map—the standard operations used to reduce resolution, such as max-pooling and strided convolutions, aggressively break this property. As Marco Manfredi and Yu Wang demonstrate, shifting an image by a single pixel can cause an object detector’s predictions to fluctuate wildly, a phenomenon that undermines the reliability of deep learning in sensitive fields like medical imaging and autonomous driving.

The root of this instability is aliasing. When a network subsamples a feature map without first removing high-frequency components, it violates the Nyquist-Shannon sampling theorem. This creates artifacts where small spatial shifts in the input lead to large, discontinuous changes in the internal representation.

The Mechanism of Signal Corruption

In a traditional architecture, a max-pooling layer selects the maximum value within a window (e.g., 2x2) and then jumps to the next window. If a bright pixel shifts from position $(x, y)$ to $(x+1, y)$, it might move from the end of one pooling window to the start of the next, or disappear from the sampled grid entirely.

Olivier J Henaff and Eero P. Simoncelli observed that this makes the "geodesics" or paths of learned representations discontinuous. To represent geometric transformations as smooth paths, a network must satisfy the Nyquist criterion: it must low-pass filter the signal before downsampling it. Without this "blurring" step, the network becomes a slave to the specific alignment of the sampling grid.

Anti-Aliasing as an Architectural Fix

To restore shift invariance, researchers have turned to anti-aliased downsampling. The core idea is simple: before any operation that reduces resolution (like a stride or pool), apply a low-pass filter (a blur kernel).

Daniel Freedman and Alex M. Bronstein applied this to chest X-ray localization, showing that incorporating low-pass filters before downsampling layers allows models to achieve state-of-the-art performance with less than 1% of the typical bounding box annotations. By stabilizing the internal representations against minor anatomical variations, the model learns to "find" diseases more consistently.

Similarly, Alexander Wong and his colleagues integrated anti-aliasing into ultra-efficient backbones like AttendNeXt. They found that this architectural modification is particularly critical for "TinyML" and edge deployment, where input data consistency cannot be guaranteed and computational budgets are tight.

From Local Jitter to Global Consistency

While basic blurring (often called "BlurPool") fixes local "jitter," it does not always solve global shift variance. Jake Lee and Junfeng Yang visualized this distinction, noting that while anti-aliasing smooths out high-frequency grid patterns, feature similarity can still degrade over larger spatial translations.

To address this, more sophisticated sampling strategies have emerged:

  • Polyphase Sampling: Davit Soselia and Jiahao Su introduced adaptive polyphase anchoring for Vision Transformers. Instead of a fixed grid, the network dynamically adjusts its sampling phase based on the input features, achieving 100% consistency under shifts.
  • Complex Wavelets: Hubert Leterme and Kévin Polisano proposed replacing early layers with complex-valued convolutions and modulus operators. This leverages the shift-invariant properties of the dual-tree complex wavelet transform to stabilize feature maps more efficiently than standard blurring.

Impact on Temporal and Perceptual Stability

The benefits of anti-aliasing extend beyond static images. In video generation, Yiran Xu and Richard Zhang used anti-aliasing in VideoGigaGAN to suppress temporal flickering. By ensuring that small shifts between frames don't cause massive changes in feature maps, the model maintains high-frequency detail without the "shimmering" artifacts common in generative video.

Furthermore, anti-aliasing is becoming a standard component in perceptual metrics. Abhijay Ghildyal and Feng Liu demonstrated that standard metrics like LPIPS are fragile to 1-pixel shifts. By redesigning the underlying networks with anti-aliased pooling, they created metrics that align more closely with human visual persistence, which does not "flip" its judgment based on imperceptible translations.

Ultimately, this research direction suggests that shift invariance should not be left to chance or data augmentation. By treating neural networks as signal processing systems and respecting the Nyquist criterion through anti-aliased downsampling, we can build models that are inherently robust to the spatial volatility of the real world.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_restoring_shift_invariance_with_anti_ali_f4eb0c3a,
  title = {Restoring Shift Invariance with Anti-Aliased Downsampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/restoring-shift-invariance-with-anti-aliased-downsampling/rdr_f4eb0c3aca7d45429e91bb6d4973c135},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_restoring_shift_invariance_with_anti_ali_f4eb0c3a_v1,
  title = {Restoring Shift Invariance with Anti-Aliased Downsampling},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/restoring-shift-invariance-with-anti-aliased-downsampling/rdr_f4eb0c3aca7d45429e91bb6d4973c135/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}