DeepN-JPEG: a deep neural network favorable JPEG-based image compression framework

Zihao Liu, Tao Liu, Wujie Wen, Lei Jiang, Jie Xu, Yanzhi Wang, Gang Quan

Introduction

In the era of the Internet of Things (IoT), edge devices like smart cameras generate vast amounts of visual data. Transmitting raw, high-resolution images to the cloud for AI processing is often prohibited by bandwidth constraints and power consumption. Consequently, lossy compression—most notably JPEG—is employed to reduce data volume.

However, standard JPEG compression is designed for the Human Visual System (HVS). It operates on the assumption that high-frequency details, which humans often cannot perceive, are redundant. Deep Neural Networks (DNNs), conversely, frequently rely on these very high-frequency features to distinguish between similar classes. Aggressive standard compression can lead to accuracy drops of up to 9%, effectively nullifying the benefits of advanced AI architectures. To address this, we present DeepN-JPEG, a framework that redesigns JPEG quantization specifically for the needs of DNNs.

Methodology: DNN-Favorable Compression

DeepN-JPEG modifies the standard JPEG pipeline by replacing HVS-centric quantization tables with tables optimized for AI performance.

The standard JPEG compression pipeline, where DeepN-JPEG introduces a custom 'Qtable' optimized for DNNs.

The framework utilizes a heuristic design flow to generate these tables:

  1. Frequency Importance Analysis: By analyzing DNN responses to different frequency components via Discrete Cosine Transform (DCT), the researchers found that a band's importance to a model is closely tied to its statistical energy (standard deviation) across the dataset.
  2. Magnitude-Based Banding: Unlike standard JPEG, which prioritizes low frequencies based on spatial position, DeepN-JPEG ranks all 64 frequency bands by their statistical significance to the target dataset.
  3. Piece-wise Linear Mapping (PLM): A mapping function categorizes these bands into "Low," "Middle," and "High" importance levels, allowing for precise tuning of the quantization table to balance file size against classification accuracy.
  4. Heuristic Design Flow: The framework samples a subset of the target dataset to calculate frequency importance and generate an optimized quantization table that can be deployed across edge devices.

Experimental Results

The framework was validated using the ImageNet dataset across several architectures, including AlexNet, VGG-16, GoogLeNet, and ResNet-50.

Comparison of Top-1 Accuracy between standard JPEG (Case 1) and DeepN-JPEG (Case 2) at different compression ratios (CR).

Key Findings

  • Efficiency Gains: DeepN-JPEG achieves approximately 3.5x higher compression rates than standard JPEG while maintaining identical classification accuracy.
  • Power Optimization: In resource-constrained IoT environments, the framework reduces communication energy consumption to just 30% of that required by standard JPEG offloading.
  • Generality: The "DNN-favorable" features identified are consistent across different model architectures, proving the framework's broad applicability.
  • The HVS vs. DNN Gap: A notable insight is that models can misclassify objects (e.g., a "junco" bird as a "robin") due to the loss of high-frequency components that are invisible to humans but critical for AI feature extraction.

Conclusion

DeepN-JPEG demonstrates that standard compression formats can be significantly improved for AI-centric workflows by optimizing the quantization logic. By aligning compression with the specific feature requirements of neural networks, the framework enables high-performance inference on edge devices with minimal overhead. Future research will investigate the impact of this approach on other tasks such as object detection and semantic segmentation.

Summary updated

BibTeX
@inproceedings{liu2018deepn,
  author = {Zihao Liu and Tao Liu and Wujie Wen and Lei Jiang and Jie Xu and Yanzhi Wang and Gang Quan},
  title = {{DeepN-JPEG: a deep neural network favorable JPEG-based image compression framework}},
  booktitle = {DAC},
  year = {2018},
  url = {https://openreview.net/forum?id=BYiF_98wDKF},
}
Figure