Log-Polar Space Convolution

Bing Su, Ji-Rong WenICLR 2022 · OpenReview

TL;DR

This paper introduces Log-Polar Space Convolution (LPSC), a novel convolutional kernel that expands the receptive field exponentially without increasing parameters. By dividing the receptive field into log-polar regions and sharing weights within these regions, LPSC naturally captures local spatial structures and improves efficiency, outperforming traditional and dilated convolutions in various vision tasks.

What Problem Does This Paper Address?

Convolutional Neural Networks (CNNs) are powerful for computer vision, but their core operation, convolution, relies on square kernels. A major challenge is increasing the "receptive field" – the area of the input image a kernel "sees" – to capture broader context. Simply making kernels larger leads to a quadratic increase in parameters, making models computationally expensive and prone to vanishing gradients in deeper layers. Existing methods like pooling or dilated convolutions have drawbacks: pooling loses information, and dilated convolutions skip pixels, potentially missing details. This paper aims to address the need for larger receptive fields with fewer parameters and a more structured approach to feature extraction.

Methods

The core innovation is the Log-Polar Space Convolution (LPSC) kernel. Instead of a uniform grid, LPSC divides the receptive field into regions based on logarithmic distance from the center and angular direction. Pixels falling into the same log-polar region share the same convolution weight. This means:

  • Exponential Receptive Field Growth: As you move further from the center, the regions become larger, allowing the kernel to cover more area without needing more parameters.
  • Structured Context: The log-polar division naturally encodes spatial relationships, giving more importance to nearby pixels and less to distant ones.
  • Parameter Efficiency: A large LPSC kernel can have significantly fewer parameters than a similarly sized conventional kernel.
  • Implementation: To integrate LPSC into existing deep learning frameworks, the authors propose log-polar space pooling. This technique effectively transforms the input feature map into a format that can be processed by standard convolutional layers, allowing LPSC to replace conventional or dilated convolutions seamlessly. They also include a "center pixel convolution" to further enhance the model's performance.

Datasets

The paper evaluates LPSC on several standard computer vision datasets:

  • Image Classification: CIFAR-10, CIFAR-100, and ImageNet.
  • Semantic Segmentation: PASCAL VOC 2012.
  • Medical Image Analysis: DRIVE (retinal vessel detection).

Key Insights and Ideas

  • Log-Polarity is Key: The paper demonstrates that a non-uniform, log-polar distribution of weights within a convolutional kernel can be more effective than uniform grids for capturing spatial context.
  • Efficiency Through Sharing: By grouping pixels into log-polar regions and sharing weights, LPSC achieves a larger receptive field with fewer parameters, making models more efficient.
  • Natural Contextual Encoding: The log-polar structure inherently prioritizes local information and gradually reduces the influence of distant pixels, mimicking how humans perceive visual information.
  • Flexibility of Implementation: The log-polar space pooling method makes LPSC easily pluggable into existing CNN architectures, allowing for direct comparison and integration.
  • Robustness to Initialization: The paper notes that LPSC shows limited sensitivity to initialization, which is a desirable trait for deep learning models.

Key Results

  • Improved Accuracy: LPSC consistently outperforms or matches conventional and dilated convolutions across various architectures (AlexNet, VGGNet, ResNet) and tasks, including image classification, segmentation, and medical image analysis. For instance, on CIFAR-100, LPSC achieved 47.31% accuracy with comparable parameters to conventional convolution.
  • Reduced Parameters: LPSC significantly reduces the number of parameters compared to conventional kernels of similar receptive field sizes.
  • Lower FLOPs: LPSC demonstrates lower Floating Point Operations (FLOPs) compared to other methods, indicating computational efficiency.
  • Enhanced Receptive Field: Visualizations show that LPSC kernels effectively enlarge the receptive field, allowing gradients to back-propagate to more input pixels.
  • Segmentation Gains: In semantic segmentation on PASCAL VOC 2012, LPSC improved the mean IoU (mIoU) of DeepLabv3+ by 1.1%.

Limitations and Future Work

  • Hyper-parameter Overhead: LPSC introduces three additional hyper-parameters (Lr, Le, g) that need to be tuned. However, the authors note that their practical ranges are limited and generally lead to good results.
  • Memory Consumption: The log-polar space pooling implementation can increase memory overhead due to the upsampled feature map.
  • Future Directions: The authors suggest that LPSC can be combined with other advanced convolution techniques like group and separable convolutions. They also mention that graph convolutions could benefit from LPSC if node distances and directions are defined. Further exploration of the elliptical kernel shape and its potential for fitting objects with different rotations and scales is also a promising avenue.

Summary updated

Abstract

Convolutional neural networks use regular quadrilateral convolution kernels to extract features. Since the number of parameters increases quadratically with the size of the convolution kernel, many popular models use small convolution kernels, resulting in small local receptive fields in lower layers. This paper proposes a novel log-polar space convolution (LPSC) method, where the convolution kernel is elliptical and adaptively divides its local receptive field into different regions according to the relative directions and logarithmic distances. The local receptive field grows exponentially with the number of distance levels. Therefore, the proposed LPSC not only naturally encodes local spatial structures, but also greatly increases the single-layer receptive field while maintaining the number of parameters. We show that LPSC can be implemented with conventional convolution via log-polar space pooling and can be applied in any network architecture to replace conventional convolutions. Experiments on different tasks and datasets demonstrate the effectiveness of the proposed LPSC.
BibTeX
@inproceedings{su2022log,
  author = {Bing Su and Ji-Rong Wen},
  title = {{Log-Polar Space Convolution}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=vEIVxSN8Xhx},
}
Figure