Breaking Translation Invariance with Coordinate Convolutions

This direction is split into other directions: translation invariance limitations, coordinate-aware convolution. Its last version stays readable and citable here.

Version 1 of 1659 papers892 conceptsfrozen 2026-10-04

Standard convolutional neural networks (CNNs) are built on a fundamental assumption: a feature learned in one part of an image should be equally recognizable in another. This property, known as translation invariance, is achieved through weight sharing—the same filter slides across the entire input. While this is a powerful inductive bias for recognizing textures and objects, it introduces a surprising "blindness." Standard CNNs struggle with tasks that require absolute spatial awareness, such as pinpointing the exact coordinates of an object or understanding its position relative to a global frame.

The Intriguing Failing of Standard Convolutions

The limitation of standard convolutions is most apparent in simple coordinate transformation tasks. As Eric Frank and colleagues demonstrated in An intriguing failing of convolutional neural networks and the CoordConv solution, a standard CNN often fails to learn the mapping between $(x, y)$ coordinates and a one-hot pixel representation. Because the filters are identical regardless of their location, the network has no inherent way of knowing "where" it is in the image.

To solve this, researchers introduced CoordConv. The mechanism is deceptively simple: instead of passing only the feature maps to the next layer, the model concatenates extra channels containing the raw Cartesian coordinates of each pixel (an $x$-channel and a $y$-channel). This allows the filters to learn functions that depend on absolute spatial position. By breaking translation invariance where necessary, the network can achieve perfect spatial generalization with far fewer parameters.

Restoring Anatomical and Global Context

In specialized domains like medical imaging, location is often as important as appearance. A lesion in the brain has different clinical implications depending on which lobe it occupies. Standard "location-blind" models can produce anatomically impossible results, such as placing a heart-specific feature in a foot scan.

To address this, Bram Platel and others developed location-sensitive architectures that inject explicit spatial coordinates and anatomical distance metrics into the network. This provides a "global anatomical context," allowing the model to distinguish between visually similar tissues based on their physical location within a standardized atlas. Similarly, Li Chen introduced the Y-net, which uses a dedicated "position path" to help segment complex intracranial arteries by leveraging their fixed anatomical positions.

This spatial awareness is also critical for physical simulations. Daniel Zoran and the DeepMind team used coordinate channels in the Visual Interaction Network to help the model localize objects and predict their future trajectories across hundreds of time steps.

Anchoring Generative Models

Generative Adversarial Networks (GANs) often rely on an accidental "secret sauce" for spatial structure: zero-padding. Rui Xu discovered that zero-padding acts as an implicit positional encoding, giving the generator a sense of where the edges are. However, this signal weakens toward the center of the image.

By replacing this implicit signal with explicit coordinate grids or sinusoidal waves, researchers have achieved much higher structural coherence. For instance, Evangelos Ntavelis showed in ScaleParty that explicit coordinate grids allow a generator to synthesize images at arbitrary resolutions while maintaining perfect object identity. Without these coordinates, the model might "double" objects or lose global symmetry when asked to generate a resolution it hasn't seen before.

Beyond Simple Cartesian Grids

The concept of coordinate injection has evolved beyond simple $x-y$ maps:

  • Fourier and Sinusoidal Embeddings: In tasks like image editing and superpixel generation, researchers like Jakob Geusen use sinusoidal positional encodings to ensure that feature embeddings are spatially constrained, leading to more compact and contiguous segments.
  • Coordinate-Based Losses: Quanziang Wang proposed Residual Moment Loss, which integrates coordinate information directly into the training objective. This penalizes the model more heavily if it predicts an object in an anatomically impossible location.
  • Relational Reasoning: Simyung Chang used Broadcasting Convolutional Networks to embed radial and Cartesian distances, enabling models to reason about the relationships between objects in complex scenes like the CLEVR dataset.

By treating spatial coordinates as a first-class citizen in the feature stack, these models move past the "narrow-sightedness" of local receptive fields, gaining a global view that is essential for reasoning, generation, and precise localization.

Papers in this version

Showing 60 of 659 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_breaking_translation_invariance_with_coo_aff37c69,
  title = {Breaking Translation Invariance with Coordinate Convolutions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/breaking-translation-invariance-with-coordinate-convolutions/rdr_aff37c69e2dd4920b67e6e1190348b39},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_breaking_translation_invariance_with_coo_aff37c69_v1,
  title = {Breaking Translation Invariance with Coordinate Convolutions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/breaking-translation-invariance-with-coordinate-convolutions/rdr_aff37c69e2dd4920b67e6e1190348b39/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}