Injecting Spatial Coordinates into Convolutional Networks

495 concepts308 papers8 proposals

Standard Convolutional Neural Networks (CNNs) possess a fundamental mathematical property: translation invariance. If an object shifts within an image, its activation pattern shifts identically through the network's layers. This behavior is hardcoded via weight sharing—applying the exact same small matrix (a kernel) across every local patch of an image.

While translation invariance is highly effective for tasks like object classification (where a cat is a cat regardless of whether it sits in the top-left or bottom-right corner), it becomes a major liability when the absolute position of a pixel dictates how it should be processed. For geometric transformations, spatially-varying distortions, or structured environments like medical imaging, local texture alone is insufficient.

To solve this, a powerful architectural pattern has emerged: explicitly injecting spatial coordinates directly into the network's feature maps. By breaking translation invariance selectively, networks gain absolute spatial awareness without sacrificing the efficiency of convolutional operations.


The Core Mechanism: Appending Coordinate Channels

The implementation of coordinate injection is remarkably simple. Consider a standard intermediate feature map of shape $(C, H, W)$, where $C$ is the number of channels, $H$ is height, and $W$ is width. A standard convolution kernel operates blindly on local $3 \times 3$ or $5 \times 5$ neighborhoods, completely unaware of where that neighborhood resides relative to the global image boundaries.

To ground the network, we instantiate fixed coordinate tensors—typically a normalized $x$-grid ranging from $[-1, 1]$ and a corresponding $y$-grid. These grids are concatenated along the channel dimension, transforming the feature map shape to $(C + 2, H, W)$.

Standard Feature Map (C x H x W)       Coordinate Grids (2 x H x W)
     [  Visual Features  ]       +       [ X-coords ]  [ Y-coords ]
                                 |
                                 v
               Augmented Feature Map ((C + 2) x H x W)

When the next convolutional layer processes this augmented tensor, its weights can directly condition on the coordinate channels. If a feature needs to be interpreted differently at the edge of an image than at the center, the network can learn a weight that multiplies the coordinate value to alter the output activation.

Researchers have extended this basic Cartesian formulation to capture more complex geometric priors. For instance, Simyung Chang et al. appended both zero-centered Cartesian coordinates and radial distance channels in Broadcasting Convolutional Network for Visual Relational Reasoning. This explicit inclusion of radial distance mitigates the inherent center-bias of standard convolutional kernels, vastly improving a network's capacity for precise spatial reasoning and object localization.


Why It Works: Three Intuitive Dimensions

The success of coordinate injection across diverse domains can be traced to three core intuitive mechanisms.

1. Modeling Spatially-Varying Degradations and Geometries

Physical imaging systems rarely treat light uniformly across a sensor. In modern smartphones equipped with under-display cameras, light diffraction patterns are highly dependent on the fixed geometry of the overlying display panel. As shown by Marcos V. Conde et al. in Real-Time Under-Display Cameras Image Restoration and HDR on Mobile Devices, standard convolutions struggle to correct these artifacts because the blur changes across the screen. By introducing coordinate channels, the restoration network applies localized corrections tailored to the pixel's exact position relative to the display structure.

Similarly, optical lenses introduce perspective distortions that scale smoothly from the center out to the boundaries. In PLADE-Net: Towards Pixel-Level Accuracy for Self-Supervised Single-View Depth Estimation With Neural Positional Encoding and Distilled Matting Loss, Juan Luis Gonzalez et al. integrated neural positional encodings to account for these lens-specific distortions, resolving geometric depth errors near image boundaries that translation-invariant filters fail to capture. When the task requires calculating precise pixel-to-pixel flow fields—such as unwarping a crumpled document—Ke Ma et al. demonstrated in DewarpNet: Single-Image Document Unwarping With Stacked 3D and 2D Regression Networks that breaking translation invariance is a prerequisite for accurate spatial mapping.

2. Restoring Global Context to Local Patches

Due to memory constraints, deep networks processing high-resolution data (such as 3D medical scans) must operate on small, isolated image patches. However, a patch showing a generic tissue boundary looks identical whether it originates from the top or bottom of an organ.

Injecting absolute coordinates restores "situational awareness" to these isolated views. Luca Lumetti leveraged this in Location Matters: Harnessing Spatial Information to Enhance the Segmentation of the Inferior Alveolar Canal in CBCTs, showing that embedding absolute 3D coordinates into patch-based networks allows the model to track exactly where a patch sits within a patient's anatomy.

This spatial context acts as a powerful anatomical prior. In tasks like pulmonary lobe segmentation, the authors of Automated Segmentation Of Pulmonary Lobes Using Coordination-Guided Deep Neural Networks demonstrated that absolute positioning provides a soft geometric constraint. This constraint prevents false positives by allowing the network to distinguish between visually identical textures based entirely on their global location in the body.

Furthermore, Matthew Baugh et al. in nnOOD: A Framework for Benchmarking Self-supervised Anomaly Localisation Methods and Zerui Zhang et al. in Spatial-aware Attention Generative Adversarial Network for Semi-supervised Anomaly Detection in Medical Image proved that coordinate awareness allows networks to exploit anatomical consistency, helping the model realize that a specific texture variation is perfectly normal in one organ location but highly anomalous in another.

3. Mitigating Representation Collisions

In implicit neural representations and grid-based architectures (like NeRFs), networks rely on spatial hashing to compress massive scenes. A common failure mode is a "hash collision," where two entirely separate regions of a scene map to the same feature index, causing global regularization techniques to accidentally erase fine foreground details.

Yizhou Li et al. addressed this in Segmentation-Guided Neural Radiance Fields for Novel Street View Synthesis by incorporating explicit positional embeddings alongside the feature grids. The coordinates serve as a unique mathematical anchor, ensuring that even if visual features collide in the hash table, the network can still separate them based on their absolute coordinates.


Generalization Beyond 2D Images

The intuition behind coordinate injection extends far beyond standard 2D image grids. Whenever a deep learning model processes data where position carries semantic meaning, coordinates can be injected to anchor local operations.

By treating coordinates as a first-class feature channel, these architectures demonstrate that breaking translation invariance is not a flaw to be avoided, but a powerful tool for grounding local computations in a global context.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_coordinate_aware_convolution_dff7dd91,
  title = {coordinate-aware convolution},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/coordinate-aware-convolution/rdr_dff7dd916be4423cb353c868a010c6b6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_coordinate_aware_convolution_dff7dd91_v1,
  title = {coordinate-aware convolution},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/coordinate-aware-convolution/rdr_dff7dd916be4423cb353c868a010c6b6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake