In dense prediction tasks like semantic segmentation, object detection, and image restoration, networks must resolve a fundamental tension: they need a large receptive field to understand global context, but they must also preserve high-resolution spatial details.
Standard convolutional neural networks (CNNs) expand their receptive field by downsampling feature maps via pooling or strided convolutions. However, aggressively reducing spatial resolution discards the precise geometric information needed to locate boundaries or identify small objects.
Dilated convolutions (also known as atrous convolutions) offer an elegant alternative. By inserting spaces ("holes") between the weights of a standard convolutional kernel, a dilated filter expands its receptive field exponentially without losing spatial resolution and without increasing the number of learnable parameters.
Yet, this architectural superpower comes with a severe side effect: the gridding artifact.
Anatomy of the Gridding Effect
To understand why gridding artifacts happen, we must look at how information propagates through stacked dilated layers.
When a convolutional kernel is dilated by a rate of $r$, it samples pixels at intervals of $r-1$. For a single layer, this sparse sampling allows the network to look across a wider area. The problem arises when we stack multiple layers that share the same uniform dilation rate.
Imagine a sequence of convolutional layers, each with a dilation rate of $r = 2$.
- Layer 1 samples every second pixel from the input image.
- Layer 2 samples every second pixel from Layer 1's feature map.
Because the sampling rate is perfectly synchronized and uniform, the deeper layers end up computing outputs from a sparse, checkerboard-like grid of input pixels. The intermediate pixels in the gaps are completely ignored.
Layer 2 Grid: X . X . X . X . X
. . . . . . . . .
Layer 1 Grid: X . X . X . X . X
. . . . . . . . .
Input Pixels: X X X X X X X X X
As noted by Panqu Wang et al. in Understanding Convolution for Semantic Segmentation, this uniform spacing creates a sparse sampling grid that fails to maintain local consistency. Because adjacent pixels do not contribute to the same higher-level representations, the network loses its grasp on continuous local structures.
This phenomenon manifests visually as high-frequency checkerboard patterns or blocky glitches in the network's activation maps. As Fisher Yu et al. demonstrate in Dilated Residual Networks, these high-frequency sampling errors act as aliasing artifacts, destroying the smooth feature representations required for dense pixel-level predictions.
The Cost of Sparsity
The consequences of the gridding effect are particularly destructive in high-precision domains:
- Small Object Disappearance: As Heqiang Huang et al. observe in YOLO-TS, if a small target falls entirely within the "holes" of a dilated grid, the network will miss it completely.
- Boundary Degradation: When content-adaptive operations or edge detection networks use sparse kernels, the gridding effect introduces noise and irregularities precisely around object boundaries, as shown by Hang Su et al. in Pixel-Adaptive Convolutional Neural Networks.
- Structural Fragmentation: For thin, continuous structures like roads or lane markings, sparse sampling causes fragmented, broken predictions because the model skips over the local pixels that bind the line together (Shao-Yuan Lo et al.).
Designing Around the Grid
To eliminate gridding artifacts, researchers have developed architectural strategies designed to restore local spatial continuity. The core intuition behind all these solutions is simple: ensure that every pixel within the receptive field eventually contributes to the final computation.
1. Interlocking Dilation Rates (The Sawtooth Pattern)
The most direct way to break the grid is to vary the dilation rates across consecutive layers so that their sampling patterns interlock.
Panqu Wang et al. introduced this concept as Hybrid Dilated Convolution (HDC). Instead of stacking layers with a constant dilation rate (e.g., $2, 2, 2$), HDC uses a progressive "sawtooth" wave of changing rates (e.g., $1, 2, 3$).
By changing the step size of each successive layer, the holes left open by one layer are covered by the next. This mathematical coordination ensures that the combined, effective receptive field covers the input area densely and continuously, completely eliminating the blind spots.
A similar philosophy underpins the Degressive Dilation Blocks proposed by Shao-Yuan Lo et al., which use a sequence of progressively decreasing dilation rates to act as a localized fill-in mechanism for narrow structures.
2. Parallel Complementary Branches
Instead of fixing the grid sequentially across layers, you can fix it simultaneously using parallel operations.
In High-Resolution Dual-Stage Multi-Level Feature Aggregation for Single Image and Video Deblurring, Stephan Brehm et al. design parallel convolution branches with complementary dilation rates. Because the branches sample offset patterns at the same time, merging their outputs ensures complete pixel coverage across a defined rectangular area, maintaining a smooth and continuous receptive field.
3. Mathematical Constraints on Dilation
Can we use dilated convolutions freely without causing artifacts? The authors of Rethinking Dilated Convolution for Real-time Semantic Segmentation proved that the gridding effect can be prevented entirely if the dilation rate is mathematically bounded by the kernel size and stride.
Specifically, keeping the dilation rate less than or equal to the kernel size divided by the stride ($r \le K/S$) guarantees continuous spatial coverage across the feature map. When architectures violate this boundary to chase a larger field-of-view, they introduce the spatial discontinuities that cause gridding.
4. Feature Smoothing and Blurring
Another approach treats gridding as a high-frequency noise problem that can be filtered out.
Daniel Merget et al. address this in Robust Facial Landmark Detection via a Fully-Convolutional Local-Global Context Network by introducing an implicit kernel convolution that blurs intermediate feature maps. By pre-processing features with smoothing kernels, they remove the high-frequency sampling noise before it can propagate through the network and disrupt global context aggregation. Similarly, Fisher Yu et al. add post-processing layers with decreasing dilation rates specifically to smooth out the final activation maps.
Summary
Dilated convolutions are incredibly powerful because they decouple a network's receptive field size from its spatial resolution. However, uniform dilation creates a rigid, sparse sampling pattern that ignores local spatial dependencies.
By understanding the geometry of this "gridding effect," modern architectures avoid it entirely—whether by coordinating dilation rates in a sawtooth pattern, applying strict mathematical constraints, or using parallel complementary branches to fill in the gaps. The result is an architecture capable of capturing massive global context while maintaining pixel-perfect local consistency.
Key Foundational Work
Understanding Convolution for Semantic Segmentation (2017) — contributed 4 concept elements to this direction.
Tree-structured Kronecker Convolutional Network for Semantic Segmentation (2018) — contributed 3 concept elements to this direction.
Smoothed Dilated Convolutions for Improved Dense Prediction (2018) — contributed 2 concept elements to this direction.