Improving Robustness of Feature Representations to Image Deformations using Powered Convolution in CNNs

Zhun Sun, Mete Ozay, Takayuki OkataniarXiv 2017 · OpenReview

Abstract

Convolutional Neural Networks (CNNs) often struggle with image deformations such as blur, noise, or perspective changes. This fragility stems from shifts in the higher-moment statistics of feature distributions that standard normalization techniques fail to correct. We propose Powered Convolution, which integrates learnable power-function non-linearities into the convolution operation. This method allows networks to automatically align shifted distributions, significantly enhancing robustness. Experimental results on large-scale benchmarks like ImageNet and Pascal VOC demonstrate substantial gains—up to a 3.3% mAP boost in object detection under deformation—with negligible computational overhead.

The Challenge: Statistical Shifts in Deformed Images

While CNNs excel at visual recognition on clean datasets, their performance often degrades sharply when encountering real-world "dataset shift." Deformations like Gaussian blur, JPEG compression, or structured noise do more than just obscure pixels; they warp the underlying statistical properties of the features the network has learned to recognize.

Standard techniques like Batch Normalization are designed to align the mean and variance (the first and second moments) of data. However, deformations frequently alter higher-moment statistics, such as skewness and kurtosis. When these "shapes" of feature distributions diverge between training and testing data, the network's filters no longer find the patterns they were optimized for.

Overview of image deformations and their impact on feature map distributions. Deformations cause shifts in the statistical 'shape' of features, which standard CNNs struggle to handle.

Methodology: Powered Convolution

To address these higher-order shifts, we introduce Powered Convolution. This approach replaces or augments the standard linear weighted sum of a convolution with a learnable power function:

$$y = \text{sgn}(x) \frac{|x|^{\alpha+1}}{\beta+1}$$

In this formulation, $\alpha$ (the exponent) and $\beta$ (the scale) are parameters learned during training. These parameters allow the network to dynamically "squash" or "stretch" feature values, effectively re-aligning the distributions of deformed images with those of clean images.

The implementation can be applied either in-channel (before the convolution) or out-channel (after the convolution). To maintain efficiency, we utilize a parameter-sharing strategy where groups of channels share the same $\alpha$ and $\beta$ values. Additionally, a specialized cosine-based learning rate scheme is employed to ensure training stability, preventing the gradient issues typically associated with power functions.

Key Insights: Distribution Alignment

The primary advantage of Powered Convolution is its ability to minimize the divergence between "clean" and "deformed" feature distributions. By visualizing the Jensen-Shannon (JS) divergence, we can see how different activation and normalization functions perform this task.

Comparison of feature distribution alignment across different normalization and activation methods. The Power function (e) effectively reshapes the distribution to minimize divergence between clean and deformed data.

As shown in the figure above, standard normalization and common activations like Tanh or Softsign often fail to significantly reduce the gap between distributions. In contrast, the Power function provides the necessary non-linearity to "map" the deformed features back into a space that the subsequent layers of the classifier can interpret correctly. This acts as a computational "shortcut," achieving what would otherwise require a much deeper and more complex standard architecture.

Experimental Results

The proposed method was validated across multiple benchmarks:

  • Generalization: On CIFAR-100, Powered Convolution reduced classification error from 51.50% to 50.21%.
  • Robustness to Noise: In ImageNet (ILSVRC-12) tests, the error rate for images with "Structured Noise" plummeted from 39.5% with standard ReLU to just 19.7%.
  • Object Detection: On the Pascal VOC 2007 task, the model achieved a 3.3% boost in mAP on deformed images compared to the reference model, despite both models performing identically on original, clean images.
  • Efficiency: On ImageNet, the accuracy improved by nearly 2% while adding only 0.04% more parameters, demonstrating the extreme efficiency of the approach.

Conclusion and Future Work

Powered Convolution offers a lightweight yet powerful solution for making CNNs more robust to the unpredictable nature of real-world visual data. By moving beyond simple mean and variance normalization and addressing higher-moment statistical shifts, we can build models that maintain high performance even under significant deformation. Future work will explore how to better preserve spatial information under extreme blur or occlusion, where pixel-wise power functions currently face limitations.

Summary updated

Abstract

In this work, we address the problem of improvement of robustness of feature representations learned using convolutional neural networks (CNNs) to image deformation. We argue that higher moment statistics of feature distributions could be shifted due to image deformations, and the shift leads to degrade of performance and cannot be reduced by ordinary normalization methods as observed in experimental analyses. In order to attenuate this effect, we apply additional non-linearity in CNNs by combining power functions with learnable parameters into convolution operation. In the experiments, we observe that CNNs which employ the proposed method obtain remarkable boost in both the generalization performance and the robustness under various types of deformations using large scale benchmark datasets. For instance, a model equipped with the proposed method obtains 3.3\% performance boost in mAP on Pascal Voc object detection task using deformed images, compared to the reference model, while both models provide the same performance using original images. To the best of our knowledge, this is the first work that studies robustness of deep features learned using CNNs to a wide range of deformations for object recognition and detection.
BibTeX
@misc{sun2017improving,
  author = {Zhun Sun and Mete Ozay and Takayuki Okatani},
  title = {{Improving Robustness of Feature Representations to Image Deformations using Powered Convolution in CNNs}},
  year = {2017},
  eprint = {1707.07830},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1707.07830},
}

Loading related content...

Figure