Log-Polar Space Convolution Layers

Bing Su, Ji-Rong WenNeurIPS 2022 · OpenReview

This NeurIPS 2022 paper introduces a clever way to give convolutional neural networks (CNNs) a "wider view" of the world without the usual computational tax. By rethinking the geometry of the convolution kernel, the authors propose a method to exponentially expand a neuron's receptive field while keeping the parameter count remarkably low.

The Receptive Field Dilemma

In computer vision, the "Local Receptive Field" (LRF) is the specific area of an image a single convolutional kernel "sees." Generally, a larger LRF is highly desirable because it allows the model to understand the relationship between distant parts of an image, providing crucial global context.

However, expanding the LRF with current methods introduces a "Goldilocks" problem:

  1. Standard Convolutions: Increasing the kernel size (e.g., from $3\times3$ to $9\times9$) causes the number of parameters to explode quadratically, leading to bloated models.
  2. Stacking Layers: Going deeper increases the theoretical receptive field, but it can lead to vanishing gradients and doesn't always result in a larger effective receptive field.
  3. Dilated Convolutions: These skip pixels to cover more ground, but the "holes" in the kernel mean they miss fine-grained details and can cause aliasing artifacts.

The authors sought a method that provides a large, continuous receptive field without the parameter bloat or loss of local detail.

Enter Log-Polar Space Convolution (LPSC)

The core innovation of this paper is the LPSC kernel, which abandons the traditional square grid in favor of a log-polar coordinate system.

Log-Polar Space Convolution (LPSC) kernel applied to an image, demonstrating the log-polar grid structure.
  • The "Bullseye" Grid: Instead of standard $X$ and $Y$ coordinates, the kernel is divided into distance levels ($L_r$) and direction levels ($L_\theta$).
  • Logarithmic Scaling: The distance between rings increases exponentially as you move away from the center. This means the kernel has high resolution at the center (where pixels are distinct) and lower resolution at the periphery (where many pixels share a single weight).
  • Weight Sharing: All pixels falling into the same "bin" (a segment of a ring) share the same learnable weight. This allows a kernel with just 13–25 parameters to cover an area that would normally require hundreds of parameters.

Practical Implementation: Log-Polar Pooling

To make this non-standard geometry compatible with modern deep learning frameworks like PyTorch, the authors developed a clever "pooling" trick.

Mapping the log-polar grid to a standard rectangular grid for conventional convolution.

They transform the input feature map into a log-polar representation, apply a standard convolution, and then map it back. For architectures like ResNet, they also propose Cross-Convolution (LPSC-CC), replacing only a percentage ($p\%$) of standard kernels with LPSC kernels to balance local detail with global context.

Key Insights and Advantages

  • Bio-inspired Prior: LPSC mimics the human fovea. Our eyes see the center of our vision in high resolution, while the periphery is blurrier but excellent at detecting broad structures and motion. LPSC applies this exact logic to neural weights.
  • Exponential Growth: Because the rings grow logarithmically, the receptive field size increases exponentially with the number of distance levels ($L_r$), not quadratically.
  • Parameter Efficiency: An LPSC kernel can achieve the same reach as a $9\times9$ standard convolution (81 parameters) using only about 19 parameters, completely avoiding the "blind spots" found in dilated convolutions.
  • Structural Encoding: By design, LPSC forces the model to learn spatial structures relative to a center point, which acts as a natural regularizer for local feature extraction.

Experimental Results

The authors evaluated LPSC across a variety of tasks and standard benchmarks, demonstrating consistent improvements:

  • Image Classification: On CIFAR-100, an AlexNet equipped with LPSC outperformed the standard version by 3.45% while using slightly fewer parameters. Replacing just a few layers in a ResNet-18 with LPSC-CC improved Top-1 accuracy on ImageNet-1K, showing that even highly optimized models benefit from this structured context.
  • Semantic Segmentation: On PASCAL VOC 2012, LPSC improved the Mean Intersection over Union (mIoU) of DeepLabv3+ by 1.1%. It proved particularly effective at replacing the largest dilated convolutions in the Atrous Spatial Pyramid Pooling (ASPP) module.
  • Efficiency: While the current implementation has some overhead, LPSC significantly reduces FLOPs (floating-point operations) compared to large-kernel standard convolutions.

Limitations and Future Work

The authors acknowledge that their current implementation using log-polar pooling is slower in terms of wall-clock time than standard convolutions, which have been hyper-optimized by hardware manufacturers for decades. Custom CUDA kernels would be needed for production-level speed. Additionally, LPSC introduces new hyperparameters (like distance/direction levels and growth rate) that require tuning, and it may not be ideal for "semantically dense" data like text where spatial, center-out priority doesn't apply.

Ultimately, LPSC serves as a powerful "plug-and-play" replacement for standard convolutions, especially in tasks where capturing a wide context is more important than uniform high-resolution processing across the entire kernel.

Summary updated

Abstract

Convolutional neural networks use regular quadrilateral convolution kernels to extract features. Since the number of parameters increases quadratically with the size of the convolution kernel, many popular models use small convolution kernels, resulting in small local receptive fields in lower layers. This paper proposes a novel log-polar space convolution (LPSC) layer, where the convolution kernel is elliptical and adaptively divides its local receptive field into different regions according to the relative directions and logarithmic distances. The local receptive field grows exponentially with the number of distance levels. Therefore, the proposed LPSC not only naturally encodes local spatial structures, but also greatly increases the single-layer receptive field while maintaining the number of parameters. We show that LPSC can be implemented with conventional convolution via log-polar space pooling and can be applied in any network architecture to replace conventional convolutions. Experiments on different tasks and datasets demonstrate the effectiveness of the proposed LPSC.
BibTeX
@inproceedings{su2022log,
  author = {Bing Su and Ji-Rong Wen},
  title = {{Log-Polar Space Convolution Layers}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=XBd2tVRfD5},
}
Figure