Learnable Cube-based Video Encryption for Privacy-Preserving Action Recognition

Yuchi Ishikawa, Masayoshi Kondo, Hirokatsu KataokaWACV 2024 · DOI

Introduction

Cloud-based video analytics offer significant computational advantages for tasks like surveillance and industrial monitoring. However, transmitting raw video data to third-party servers introduces severe privacy risks. Traditional solutions often involve a trade-off: anonymization techniques like blurring or downsampling protect privacy but degrade AI recognition accuracy, while standard encryption prevents AI processing altogether without decryption. This paper introduces a novel framework that achieves privacy-preserving action recognition with zero loss in accuracy.

Methodology

The proposed approach consists of two integrated components: Learnable Cube-based Video Encryption (LCVE) and ViT Scrambling.

Learnable Cube-based Video Encryption (LCVE)

Unlike frame-based encryption, LCVE treats video as a 4D spatio-temporal volume. The process involves two primary stages:

  1. Cube Shuffling: The video is partitioned into spatio-temporal cubes (e.g., $2 \times 16 \times 16$ blocks), which are then randomly shuffled throughout the video volume.
  2. Pixel Shuffling: Within each individual cube, RGB pixel values are shuffled based on a secret key.

This multi-level shuffling results in a video that appears as visual noise to human observers and unauthorized AI models, while significantly expanding the key space to approximately $3.5 \times 10^{8560}$, rendering brute-force attacks computationally infeasible.

The LCVE encryption process: dividing video into spatio-temporal cubes, shuffling cubes, and shuffling pixel values.

ViT Scrambling

The core innovation lies in the synergy between LCVE and Vision Transformers (ViTs). Because ViTs process images as sequences of patches, the researchers developed a method to "scramble" the model's internal weights to match the encrypted input. By rearranging positional embeddings and linear projection weights according to the encryption key, the model's first-layer output remains mathematically identical to that of a model processing unencrypted video. This eliminates the need for retraining on encrypted data.

Experimental Results

The framework was evaluated across seven diverse datasets, including Kinetics400, SSV2, and privacy-focused benchmarks like the KTH Dataset.

Comparison of visual quality and performance metrics between plain video, extreme low-resolution, and the proposed LCVE method.

The results demonstrate that LCVE maintains 80.2% accuracy on the Kinetics400 dataset—identical to the performance on plain video. Simultaneously, privacy protection is significantly enhanced; actor identification accuracy on the KTH dataset dropped from 98.8% to just 4.0% when using LCVE. This outperforms traditional "Extreme Low-resolution" methods, which typically suffer from substantial accuracy drops (often 20-30%) while providing weaker privacy guarantees.

Conclusion

By exploiting the permutation invariance of Vision Transformers, the LCVE framework provides a robust solution for secure cloud-based video analytics. While currently optimized for classification tasks and Transformer architectures, the method sets a new benchmark for "zero-utility-loss" privacy-preserving AI. Future work may extend these principles to dense-prediction tasks like segmentation and other neural architectures.

Summary updated

BibTeX
@inproceedings{ishikawa2024learnable,
  author = {Yuchi Ishikawa and Masayoshi Kondo and Hirokatsu Kataoka},
  title = {{Learnable Cube-based Video Encryption for Privacy-Preserving Action Recognition}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
  doi = {10.1109/wacv57701.2024.00685},
}
Figure