The defining feature of standard convolutional neural networks (CNNs) is weight sharing. As a convolutional kernel slides across an image, it applies the exact same weights to every local neighborhood. This design enforces translation invariance (or shift invariance): a feature detector that recognizes a specific pattern—such as an edge, a texture, or an object part—will produce the same activation value regardless of whether that pattern appears in the top-left corner or the bottom-right corner of the frame.
For image classification, translation invariance is incredibly powerful. A cat is a cat whether it sits on the left or the right side of an image. However, for tasks that require precise localization, spatial reasoning, or instance differentiation, this invariance becomes a fundamental limitation. Because standard convolutions treat all locations identically, they are inherently agnostic to absolute spatial coordinates.
To solve tasks where position matters, researchers have developed a variety of techniques to break translation invariance, injecting explicit or learned spatial coordinate awareness back into convolutional architectures.
The Blind Spots of Translation Invariance
When a network is completely translation-invariant, it struggles with two broad classes of problems: distinguishing between visually identical instances and respecting absolute spatial constraints.
1. The Instance Segmentation Dilemma
In instance segmentation, a network must not only identify object categories but also isolate individual object instances. Imagine an image containing two identical, adjacent apples. A pixel on the first apple and a pixel on the second apple might share identical local visual neighborhoods.
Because standard fully convolutional networks (FCNs) are translation-invariant, they output identical semantic labels for identical visual patterns. As discussed by Jifeng Dai et al. in Instance-Sensitive Fully Convolutional Networks, this property prevents standard FCNs from distinguishing between individual instances of the same class. Without location-aware features, the network tends to group all pixels of the same category together. David Novotný et al. formalize this in Semi-convolutional Operators for Instance Segmentation, noting that assigning unique instance identifiers to pixels is impossible unless the architecture has a mechanism to differentiate spatially distinct but visually identical objects.
2. Absolute Spatial and Domain Constraints
In many specialized domains, an object's absolute position provides vital context. For example, in medical imaging, anatomical structures are not randomly distributed; they occupy fixed, predictable locations within the body.
B.T. Thomas Yeo et al. demonstrated this phenomenon in Different scaling of linear models and deep learning in UKBiobank brain images versus machine-learning datasets. They found that because brain scans are typically registered to a standard spatial atlas, the spatial flexibility and translation invariance of CNNs provide little to no advantage over simple linear models. When networks lack explicit coordinate awareness, they can make anatomically impossible predictions, such as misclassifying pulmonary lobes simply because they cannot ground their features in a global coordinate system (Wenjia Wang et al. in Automated Segmentation Of Pulmonary Lobes Using Coordination-Guided Deep Neural Networks).
Mechanisms for Restoring Spatial Awareness
To overcome these limitations, modern architectures introduce coordinate awareness through three primary mechanisms: explicit coordinate injection, position-sensitive score maps, and location-conditioned dynamic kernels.
Explicit Coordinate Injection
The most direct way to break translation invariance is to append coordinate channels (such as $x$ and $y$ coordinate grids) directly to the network's feature maps. This approach, often referred to as Coordinate Convolution (CoordConv), ensures that a sliding filter's input changes depending on its exact position in the image.
Standard Convolution:
[Feature Map] ---> [Conv Layer] ---> [Translation-Invariant Output]
Coordinate Convolution:
[Feature Map] ──┐
├──> [Concat] ---> [Conv Layer] ---> [Spatially Aware Output]
[X, Y Grids] ──┘
By hardcoding these coordinates, the network can easily learn functions that depend on absolute position. For instance, Ke Ma leverages this in DewarpNet: Single-Image Document Unwarping With Stacked 3D and 2D Regression Networks to map pixels from a distorted document back to a flat target coordinate space. Similarly, to improve the synthesis of complex geometric structures like human hands, Mehran Hosseini et al. introduced GeoPos: A Minimal Positional Encoding for Enhanced Fine-Grained Details in Image Synthesis Using Convolutional Networks, which injects a single randomly-shifted coordinate channel to break symmetry while preserving training stability.
Position-Sensitive Score Maps and Embeddings
Instead of hardcoding absolute coordinates as inputs, some architectures force the network to learn intermediate representations that vary continuously across space.
In Fully Convolutional Instance-Aware Semantic Segmentation, Yi Li et al. introduced "position-sensitive inside/outside score maps." Instead of generating a single score map for an object category, the network generates multiple maps corresponding to relative positions (e.g., the top-left or bottom-right of an instance). This allows a single pixel to simultaneously express a high foreground score for one object instance and a background score for an adjacent instance, effectively bypassing the limitations of standard shared weights.
Location-Conditioned Dynamic Kernels
A third paradigm completely abandons static weight sharing in favor of generating unique convolutional kernels dynamically based on spatial location.
This strategy is central to the SOLO framework proposed by Xinlong Wang et al. in SOLO: Segmenting Objects by Locations. The authors reformulate instance segmentation by classifying pixels into discrete "instance categories" defined entirely by an object's center location and size.
In the follow-up work, SOLOv2: Dynamic and Fast Instance Segmentation, Xinlong Wang splits the task into a mask feature branch and a kernel prediction branch. The kernel branch predicts customized weights conditioned on specific spatial coordinates, allowing the model to isolate individual instances dynamically. The same principle is utilized by Shi-Xue Zhang et al. in Kernel Proposal Network for Arbitrary Shape Text Detection, where instance-specific dynamic kernels prevent adjacent text blocks from bleeding into a single mask.
Why It Works: Breaking the Symmetry
Injecting coordinate awareness works because it breaks the mathematical symmetry of weight sharing. In a standard convolutional layer, the response $y$ at position $(i,j)$ is computed as:
$$y_{i,j} = \sum_{m,n} w_{m,n} \cdot x_{i+m, j+n}$$
Because the kernel weights $w$ do not depend on $i$ or $j$, the operation cannot distinguish between identical input patches at different locations. By introducing spatial coordinates—whether via input grids, position-sensitive channels, or location-conditioned weights—the operation effectively transforms into a spatially variant function:
$$y_{i,j} = f(x, i, j)$$
This simple modification preserves the local efficiency and inductive bias of convolutions while granting the network a global frame of reference, enabling it to successfully navigate tasks that demand strict spatial accountability.
Key Foundational Work
Fully Convolutional Instance-Aware Semantic Segmentation (2017) — contributed 1 concept elements to this direction.
Instance-Sensitive Fully Convolutional Networks (2016) — contributed 1 concept elements to this direction.