Investigating Image Applications Based on Spatial-Frequency Transform and Deep Learning Techniques

Qinkai Zheng, Han Qiu, Gérard Memmi, Isabelle BlocharXiv 2020 · arXiv

Abstract

This research investigates the synergy between classical spatial-frequency transforms and modern deep learning architectures to address critical challenges in image processing. The study focuses on two primary applications: improving JPEG compression efficiency and enhancing the adversarial robustness of neural networks. By leveraging the Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT), the proposed methods achieve significant data reduction in image transmission and provide a robust defense against sophisticated adversarial attacks. Specifically, the Block Artifact Removing Convolutional Neural Network (BARCNN) enables a 40% reduction in JPEG data, while the WavExt defense mechanism maintains high classification accuracy under attack.

Introduction

While deep learning has revolutionized computer vision, most state-of-the-art models operate exclusively in the spatial domain, treating images as raw pixel grids. This approach often overlooks the rich structural information available in the frequency domain. This paper argues that integrating frequency-domain techniques—such as DCT and Wavelets—can unlock new levels of performance and security. We present a hybrid framework that utilizes these transforms to optimize JPEG compression and secure models against adversarial perturbations.

Enhanced JPEG via DC Recovery (BARCNN)

Standard JPEG compression utilizes the Discrete Cosine Transform to convert 8x8 pixel blocks into one DC coefficient (representing average brightness) and 63 AC coefficients (representing detail). To improve efficiency, we propose a method that omits the transmission of most DC coefficients.

System Architecture

As illustrated in the pipeline below, the sender transmits only four corner DC coefficients for the entire image along with all AC coefficients. This strategy reduces the transmitted data volume by approximately 40%.

Proposed end-to-end pipeline for JPEG compression and DC recovery using BARCNN.

At the receiver's end, the missing DC values are initially estimated using a traditional prediction algorithm based on spatial correlation between blocks. However, this often results in visible "blocky" artifacts. To mitigate this, a 24-layer Block Artifact Removing Convolutional Neural Network (BARCNN) is employed. BARCNN uses residual learning to refine the pre-recovered DC values and eliminate artifacts, restoring the image to high fidelity.

Visualizing the DC Component

The concept of isolating the DC component is central to this data-saving technique. Figure 3 provides a 3D visualization of pixel values within a single block, highlighting the "base" average intensity (DC) that our method seeks to reconstruct at the receiver's end.

Visualization of image pixel values in an 8x8 block (a) and the identification of the DC component (b).

Wavelet Transformation Defense (WD-WavExt)

To address the vulnerability of deep neural networks to adversarial attacks, we introduce WavExt (Wavelet Extension). Adversarial noise typically manifests as high-frequency perturbations that trick spatial-domain filters.

The WavExt method enhances robustness through two key phases:

  1. Training Phase: Models are trained on a 6-channel input tensor. In addition to the standard RGB channels, three extra channels representing "low-frequency structures" extracted via DWT are included. This forces the network to prioritize stable structural features over transient pixel noise.
  2. Inference Phase: A non-differentiable Wavelet Denoising (WD) step, utilizing the BayesShrink algorithm, is applied. The non-differentiable nature of this step creates "obfuscated gradients," significantly hindering an attacker's ability to compute effective adversarial perturbations.

Experimental Results

The proposed methods were evaluated using standard benchmarks including LFW, CIFAR-10, and various compression datasets (BSDS100, Urban100).

  • Compression Performance: The BARCNN-based recovery achieved a Peak Signal-to-Noise Ratio (PSNR) exceeding 31 dB while using only 60% of the original JPEG data. The neural network refinement provided a 1.1 dB improvement over classical prediction methods.
  • Adversarial Robustness: The WD-WavExt defense demonstrated exceptional resilience against eight major attack types. Notably, against the Carlini-Wagner (CW) attack, which reduced undefended model accuracy to 0%, our method maintained a 97% accuracy rate. It also proved effective against gradient-free attacks like BPDA.

Conclusion

This research demonstrates that frequency-domain transforms remain highly relevant in the era of deep learning. By treating AI as a "frequency interpolator," we can significantly compress image data without sacrificing quality. Furthermore, structural hints provided by Wavelet transforms offer a powerful defense against adversarial exploitation. Future work will explore more aggressive AC coefficient removal and the distribution of adversarial noise across specific frequency sub-bands to further optimize these hybrid systems.

Summary updated

Abstract

This is the report for the PRIM project in Telecom Paris. This report is about applications based on spatial-frequency transform and deep learning techniques. In this report, there are two main works. The first work is about the enhanced JPEG compression method based on deep learning. we propose a novel method to highly enhance the JPEG compression by transmitting fewer image data at the sender's end. At the receiver's end, we propose a DC recovery algorithm together with the deep residual learning framework to recover images with high quality. The second work is about adversarial examples defenses based on signal processing. We propose the wavelet extension method to extend image data features, which makes it more difficult to generate adversarial examples. We further adopt wavelet denoising to reduce the influence of the adversarial perturbations. With intensive experiments, we demonstrate that both works are effective in their application scenarios.
BibTeX
@misc{zheng2020investigating,
  author = {Qinkai Zheng and Han Qiu and Gérard Memmi and Isabelle Bloch},
  title = {{Investigating Image Applications Based on Spatial-Frequency Transform and Deep Learning Techniques}},
  year = {2020},
  eprint = {2004.02756},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2004.02756},
}
Figure