A Frequency-Domain Convolutional Neural Network Architecture Based on the Frequency-Domain Randomized Offset Rectified Linear Unit and Frequency-Domain Chunk Max Pooling Method

Jinhua Lin, Lin Ma, Jingxia CuiIEEE Access 2020 · DOI

Abstract

This paper proposes a full frequency-domain convolutional neural network (FCNN) architecture designed to explore deep learning theory within the frequency domain. By introducing a Frequency-Domain Randomized Offset Rectified Linear Unit (FRReLU) and a Frequency-Domain Chunk Max Pooling (Fcmp) method, the architecture overcomes the complexity of forward and backward pipelines in frequency-domain training. FRReLU mitigates gradient issues, while Fcmp preserves spatial context. Results using ResNet-50 show a significant reduction in error rates (Top-1 error from 24.90% to 17.95%) and a 13.03x acceleration ratio compared to cuDNN, demonstrating improved accuracy without sacrificing time performance.

Introduction

Convolutional Neural Networks (CNNs) are the backbone of modern computer vision, yet their computational demands are substantial, primarily due to time-domain convolutions. While the Fast Fourier Transform (FFT) can accelerate these operations by converting convolutions into point-wise multiplications, most existing "FFT-based" models are hybrid. They frequently toggle between time and frequency domains to apply non-linear activations and pooling, creating a significant memory bandwidth bottleneck. This paper introduces a "full" frequency-domain architecture that eliminates these transitions, maintaining data in the Fourier space from input to the final classification layers.

Methodology

The proposed FCNN architecture relies on two novel representation strategies to maintain stability and accuracy within the frequency domain:

1. FRReLU (Frequency-Domain Randomized Offset Rectified Linear Unit)

Standard activation functions like ReLU do not translate directly to the complex-valued frequency domain. FRReLU is an unsaturated activation function specifically designed for the complex plane. It processes positive and negative semi-axis functions independently and utilizes randomized offsets to introduce non-linearity. This design is crucial for preventing the vanishing and exploding gradient phenomena that typically plague frequency-domain training.

2. Fcmp (Frequency-Domain Chunk Max Pooling)

Traditional frequency-domain pooling often involves simple truncation of high-frequency components, which discards vital spatial information. Fcmp addresses this by dividing the frequency matrix into "chunks" and performing max pooling within each. This "chunk decomposition mechanism" allows the network to capture coarse-grained location information and feature strength, significantly improving classification performance.

Comparison of the proposed FCNN pipeline (III) against standard cuDNN (I) and fbFFT (II) architectures. The FCNN maintains data in the frequency domain throughout the convolutional layers, performing IFFT only before the final fully connected (FC) layer.

Experimental Results

The FCNN was evaluated using the ResNet-50 backbone on the ImageNet and MetData-1 datasets.

  • Accuracy Boost: The proposed architecture significantly outperformed existing frequency-domain models. On ImageNet, it reduced the top-1 error from 24.90% to 17.95% and the top-5 error from 12.85% to 9.23%.
  • Massive Speedup: In high-bandwidth scenarios (batch size 128), the FCNN achieved an acceleration ratio of 13.03x compared to the NVIDIA cuDNN reference model. This speedup stems from avoiding repeated FFT/IFFT cycles in every layer.
  • Stability: Testing on the MetData-1 dataset confirmed that the FCNN achieves high classification accuracy with minimal variance (average difference of 0.18), proving its robustness compared to competitors like koCNN and fbFFT.

Conclusion

By integrating FRReLU and Fcmp, this research demonstrates that it is possible to train deep learning models entirely in the frequency domain without sacrificing accuracy. The FCNN architecture offers a high-performance alternative to traditional time-domain models, particularly for deep networks where the overhead of domain switching is most pronounced. Future work will focus on optimizing chunk sensitivity and scaling the architecture to even larger datasets.

Summary updated

Abstract

It is of great importance to construct a convolutional neural network architecture in the frequency domain to explore the theory of deep learning in the frequency domain. However, due to the complexity of the construction mechanism of the forward and backward pipelines needed to train the convolutional neural network in the frequency domain, higher requirements are put forward for the representation strategy of the frequency-domain activation function and pooling method in the forward and backward pipelines. Therefore, to construct a full frequency-domain convolutional neural network architecture, it is necessary to construct a frequency-domain representation strategy with a high classification accuracy and excellent time performance. In this paper, based on a chunk decomposition mechanism and the construction principle of the frequency-domain unsaturated activation function, a frequency-domain convolutional neural network architecture is proposed. Two important representation strategies are introduced into the frequency-domain forward/backward pipeline: a frequency-domain randomized offset rectified linear unit and a frequency-domain chunk max pooling method. The former can alleviate the vanishing and exploding gradient phenomena in the frequency-domain forward/backward pipeline and ensure the convergence of the convolutional neural network architecture in the frequency-domain training stage; the latter can capture the partial location information and characteristic strength of the frequency-domain neurons and improve the classification performance of the convolutional neural network in the frequency domain. This full frequency-domain convolutional neural network architecture improves the training accuracy of the convolutional neural network in the frequency-domain pipeline. The results show that on the basis of ResNet-50 as the backbone framework, an NVIDIA GeForce CUDA(Compute Unified Device Architecture) as the training pipeline, and $4\times 4$ as the activation block size of the third-level output neuron's characteristic parameter matrix, the convolutional neural network architecture proposed in this paper can lower the top-1 error from 24.90% to 17.95%, the top-5 error from 12.85% to 9.23%. Furthermore, when the batch size is equal to 128 (in the worst-case bandwidth usage scenario), the acceleration ratio of the proposed architecture can still reach 13.0375 by selecting cuDNN as the reference model. Under the same backbone framework, the proposed architecture is tested on the MetData-1 dataset, and the classification accuracy can reach the maximum value; that is, the average difference is merely 0.18. This finding shows that the proposed architecture can improve the accuracy of the deep learning-based frequency-domain convolutional neural network model without reducing the time performance and expand the frequency-domain representation strategy of the frequency-domain activation function and pooling method.
BibTeX
@inproceedings{lin2020frequency,
  author = {Jinhua Lin and Lin Ma and Jingxia Cui},
  title = {{A Frequency-Domain Convolutional Neural Network Architecture Based on the Frequency-Domain Randomized Offset Rectified Linear Unit and Frequency-Domain Chunk Max Pooling Method}},
  booktitle = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.2996250},
}
Figure