Adversarial Attacks on Deep Learning-based Video Compression and Classification Systems

Jung-Woo Chang, Mojan Javaheripi, Seira Hidano, Farinaz Koushanfar

TL;DR

RoVISQ is the first systematic framework for adversarial attacks targeting deep learning-based video compression and downstream classification. By manipulating the fundamental Rate-Distortion (R-D) relationship, RoVISQ can stealthily degrade video quality, skyrocket network bandwidth requirements, and cause downstream activity recognition models to fail—all while remaining robust enough to survive the compression process itself.

What Problem Does This Paper Address?

Video accounts for over 80% of internet traffic, making efficient compression essential for everything from Netflix streaming to autonomous vehicle navigation. Recently, Deep Neural Networks (DNNs) have begun replacing traditional codecs (like H.264/H.265) because they offer superior performance.

However, DNNs are notoriously vulnerable to adversarial perturbations—tiny, calculated noises that fool models. While researchers have attacked video classification before, these attacks usually fail in the real world because the video compression pipeline acts as a natural "denoiser," stripping away the adversarial noise before it reaches the target.

This paper addresses two major gaps:

  1. The vulnerability of the compression pipeline itself: Can an attacker break the efficiency of the encoder?
  2. Compression-surviving attacks: Can an attacker craft noise that is robust enough to pass through a compressor and still fool a downstream classifier?
Fig. 1: High-level view of (a) a live video streaming and classification system, and (b) RoVISQ attacks performed by in- jecting adversarial perturbations to the front-end video sources. To demonstrate the adversarial effect, we present the peak signal-to-noise ratio (PSNR), bit-rate, and an example outcome of video activity recognition.

Methods

The authors introduce RoVISQ (Reduction of Video Service Quality), which treats adversarial attack generation as a unified optimization problem.

The R-D Attack Framework

Standard video compression aims to minimize the number of bits ($R$) used while minimizing distortion ($D$). RoVISQ flips this logic. It solves for perturbations ($\Delta$) that maximize a new loss function:

  • Video Quality Attack: Increases $D$ while keeping $R$ constant (adding visible artifacts).
  • Bandwidth Attack: Increases $R$ while keeping $D$ constant (clogging the network).
  • RD Attack: Simultaneously increases both, leading to a total denial-of-service or severe quality degradation.
The effect of RoVISQ attacks on the R-D curve, shifting the performance away from the optimal benign state (Point A).

Universal and Robust Perturbations

To make these attacks practical for live streaming, the authors developed universal perturbations. Instead of calculating noise for every frame (which is too slow), they train a single 3D noise pattern offline. This pattern is:

  • Input-invariant: Works on any video content.
  • GOP-aware: Accounts for the "Group of Pictures" structure used in video coding, ensuring the noise propagates through temporal dependencies rather than being filtered out.
  • Parameter-agnostic: Effective across different compression ratios ($\lambda$) and even different model architectures (black-box attacks).
Fig. 5: Overview of system components for the proposed RoVISQ attacks on video compression and classification.

Datasets

The study utilizes several standard benchmarks to validate the attacks:

  • Vimeo-90K: Used for training the victim video compression models.
  • HEVC Standard Test Sequences (Classes B, C, D): Used to evaluate R-D performance across different resolutions (from 240p to 1080p).
  • UCF-101 & 20BN-JESTER: Used to evaluate the success rate of attacks against downstream human action and hand gesture recognition models.

Key Insights and Ideas

  • Compression is not a Defense: The common assumption that lossy compression "cleans" adversarial noise is false if the attacker incorporates the compression model into their optimization loop.
  • Temporal Propagation as a Weapon: Video codecs rely on previous frames to predict current ones. RoVISQ exploits this by injecting noise that "snowballs" through the temporal chain, making the attack more efficient.
  • Transferability: Attacks trained on one DNN-based compressor (like DVC) successfully transfer to others (like FVC or HLVC) and even degrade traditional non-DNN codecs like H.265.
  • Stealthiness: Despite the massive impact on bit-rate and classification, the perturbations remain largely imperceptible to the human eye before compression.

Key Results

  • Quality Degradation: RoVISQ reduced the Peak Signal-to-Noise Ratio (PSNR) by up to 5.6dB. A user study with 55 participants confirmed that these attacks frequently lead to "highly perceptible noise" that would cause users to abandon a service.
  • Bandwidth Inflation: The bandwidth attack increased the bit-rate by up to 2.4$\times$. In a real-world scenario, this could trigger constant buffering or force a stream to drop from 1080p to 360p.
  • Classifier Subversion: RoVISQ achieved over a 90% success rate in fooling downstream classifiers (e.g., making a "Basketball" video be recognized as "Yo Yo"), even after the video was compressed.
  • Defense Resilience: The attacks remained effective against common defenses, including adversarial training, video denoising (ViDeNN), and JPEG pre-processing.

Limitations and Future Work

  • Digital-Only: Currently, the perturbations must be injected digitally (e.g., via malware on a camera or a man-in-the-middle attack). The authors suggest exploring physical-world attacks, such as using modulated LED lights to inject noise directly into a camera sensor.
  • Defense Gap: The paper demonstrates that current defenses are inadequate. Developing a compression-aware adversarial defense that doesn't significantly tank coding efficiency remains an open challenge.
  • QoE Metrics: While PSNR and bit-rate are standard, human "Quality of Experience" is subjective. Future work could involve more complex perceptual models to make attacks even stealthier.

Summary updated

Abstract

Video compression plays a crucial role in video streaming and classification systems by maximizing the end-user quality of experience (QoE) at a given bandwidth budget. In this paper, we conduct the first systematic study for adversarial attacks on deep learning-based video compression and downstream classification systems. Our attack framework, dubbed RoVISQ, manipulates the Rate-Distortion ($\textit{R}$-$\textit{D}$) relationship of a video compression model to achieve one or both of the following goals: (1) increasing the network bandwidth, (2) degrading the video quality for end-users. We further devise new objectives for targeted and untargeted attacks to a downstream video classification service. Finally, we design an input-invariant perturbation that universally disrupts video compression and classification systems in real time. Unlike previously proposed attacks on video classification, our adversarial perturbations are the first to withstand compression. We empirically show the resilience of RoVISQ attacks against various defenses, i.e., adversarial training, video denoising, and JPEG compression. Our extensive experimental results on various video datasets show RoVISQ attacks deteriorate peak signal-to-noise ratio by up to 5.6dB and the bit-rate by up to $\sim$ 2.4$\times$ while achieving over 90$\%$ attack success rate on a downstream classifier. Our user study further demonstrates the effect of RoVISQ attacks on users' QoE.
BibTeX
@inproceedings{chang2023adversarial,
  author = {Jung-Woo Chang and Mojan Javaheripi and Seira Hidano and Farinaz Koushanfar},
  title = {{Adversarial Attacks on Deep Learning-based Video Compression and Classification Systems}},
  booktitle = {NDSS},
  year = {2023},
  url = {https://openreview.net/forum?id=fNW-IjIJ1E},
}
Figure