Restoring Translation Invariance in Deep Vision Architectures

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 299 papers119 conceptsfrozen 2026-10-04

It is a foundational tenet of computer vision that Convolutional Neural Networks (CNNs) are translation invariant. The logic seems sound: because convolutions share weights across all spatial locations, the network should recognize a "cat" regardless of whether it is in the top-left or bottom-right of an image. However, a growing body of evidence reveals that this invariance is largely an illusion.

As Aharon Azulay demonstrated in Why do deep convolutional networks generalize so poorly to small image transformations?, modern CNNs are surprisingly brittle. Shifting an image by a single pixel—a change imperceptible to humans—can cause a state-of-the-art classifier to flip its prediction entirely. This fragility isn't just a quirk of small models; Ofir Shifman found in Lost in Translation that even models trained on billions of images remain highly sensitive to these minute spatial perturbations.

The Root Cause: Aliasing and the Sampling Theorem

The reason these networks fail is rooted in basic signal processing. To build deep hierarchies, networks must reduce the spatial resolution of feature maps using downsampling operations like strided convolutions and max pooling.

According to the Shannon-Nyquist sampling theorem, to accurately represent a signal after downsampling, you must first remove high-frequency components that exceed the new "Nyquist" limit. If you don't, these frequencies "alias"—they fold back into lower frequencies, creating artifacts that depend entirely on the signal's exact alignment with the sampling grid.

In a CNN, max pooling and strided convolutions act as "sub-samplers" that ignore this rule. As Timothy E. Wang notes in Translation Insensitive CNNs, these operations violate the sampling theorem, introducing aliasing that makes internal representations shift-variant. When an object moves by one pixel, the "grid" of the pooling layer samples different values, leading to a completely different feature vector.

The Fragility of Transformers

The shift-invariance problem is even more pronounced in Vision Transformers (ViTs). Unlike CNNs, which have a local inductive bias, ViTs begin by "patchifying" an image into a fixed grid. Davit Soselia and colleagues point out in Reviving Shift Equivariance in Vision Transformers that this initial tokenization, combined with absolute positional embeddings, anchors the model to a specific coordinate system.

If an object moves slightly, it may be split across different patches, fundamentally changing the input tokens. Yanghao Li and the authors of MViTv2 highlight that this lack of spatial stability hinders a model's ability to generalize across different layouts, making them less robust than their convolutional counterparts in tasks like object detection.

Engineering Invariance: Anti-Aliasing and Polyphase Sampling

To fix this, researchers are looking back at classical signal processing. The most common remedy is anti-aliasing: applying a low-pass filter (like a Gaussian blur) before every downsampling step. Guojun Lu proposed DAB-pool, which uses adaptive blurring to suppress the high-frequency noise that causes classification flips.

A more sophisticated approach involves Polyphase Sampling. Instead of picking one fixed grid to sample from, polyphase methods consider all possible shifts. Sourajit Saha and Tejas Gokhale introduced TIPS (Translation Invariant Polyphase Sampling), a learnable pooling layer that explicitly learns to "undo" shifts by selecting the most stable representation across different phases. Similarly, Renan A. Rojas-Gomez developed adaptive modules that make ViT tokenization data-dependent, achieving 100% consistency under circular shifts.

Why It Matters: Precision and Safety

This isn't just a theoretical concern; it has life-or-death implications in specialized fields:

  • Medical Imaging: In Localization with Limited Annotation, Daniel Freedman and Alexander M. Bronstein show that lack of shift-invariance makes it difficult to precisely locate diseases in X-rays, as the predicted coordinates jitter with minor patient movements.
  • Object Detection: Yu Wang demonstrated that a 1-pixel shift can cause an object detector to drop a bounding box entirely, a critical failure for autonomous systems.
  • Sub-pixel Precision: Real-world motion is continuous, not discrete. Hagay Michaeli and Tomer Michaeli argue in Alias-Free Convnets that models must be invariant not just to integer-pixel shifts, but to fractional shifts. By using polynomial activations and alias-free filters, they show it is possible to maintain stability even when an object moves by half a pixel.

Ultimately, the goal of this research direction is to move away from "photographer's bias"—the tendency of datasets to have centered, well-aligned objects—and toward architectures that possess a fundamental, geometric understanding of the visual world. By respecting the laws of sampling and signal reconstruction, we can build models that are as stable and reliable as the human visual system.

Papers in this version

Showing 60 of 99 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 94 papers
  2. Version 1 · 2026-10-04 · 99 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_spatial_translation_sensitivity_bc094337,
  title = {spatial translation sensitivity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/spatial-translation-sensitivity/rdr_bc094337a39b4ecb9ca7a3bf8ae23613},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_restoring_translation_invariance_in_deep_bc094337_v1,
  title = {Restoring Translation Invariance in Deep Vision Architectures},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/restoring-translation-invariance-in-deep-vision-architectures/rdr_bc094337a39b4ecb9ca7a3bf8ae23613/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}