Combining Contrastive and Supervised Learning for Video Super-Resolution Detection

Viacheslav Meshchaninov, Ivan Molodetskikh, Dmitriy S. VatolinarXiv 2022 · DOI · OpenReview

Abstract

As Super-Resolution (SR) technology advances, the ability to distinguish between native-resolution video and artificially upscaled content has become a critical forensic challenge. Modern AI-based SR methods leave subtle traces that are often obliterated by standard video compression. This paper presents a robust forensic framework that combines contrastive and supervised learning to detect these "fake" high-resolution videos. By utilizing a hybrid loss function and a temporal-aware architecture, the proposed model maintains high detection accuracy even in heavily compressed scenarios, significantly outperforming previous state-of-the-art methods.

Introduction

The proliferation of high-definition displays has driven a massive demand for 4K and 8K content. To meet this demand, Super-Resolution (SR) techniques—particularly those powered by deep learning—are frequently used to upscale lower-resolution footage. While beneficial for consumer entertainment, this capability creates a forensic vulnerability. Dishonest vendors can misrepresent upscaled content as native high-resolution footage, leading to legal and ethical concerns in stock photography, evidence authentication, and consumer electronics marketing.

Traditional forensic methods typically look for mathematical artifacts left by simple interpolation (e.g., bicubic upscaling). However, modern neural-network-based SR methods produce much more sophisticated textures. Furthermore, the standard pipeline for video distribution involves heavy compression (such as H.264), which acts as a "forensic bleach," removing the high-frequency micro-traces that standard algorithms rely on for detection.

Methodology

Architectural Design

The researchers developed a deep-learning model based on a ResNet-50 feature encoder. Recognizing that SR artifacts often manifest as temporal inconsistencies between frames, the authors modified the first layer of the network. Instead of the standard 3-channel (RGB) input for a single image, the model accepts two consecutive frames (6 channels). This allows the network to capture motion-related traces and temporal artifacts that are absent in native-resolution sequences.

The Hybrid Loss Function

The core innovation of this research lies in its training strategy, which employs three distinct loss functions to ensure the model learns robust, discriminative features:

  1. Cross-Entropy Loss ($L_{CE}$): This provides the standard supervised signal, training the model to classify inputs into "Real" (native) or "Fake" (SR-upscaled) categories.
  2. Triplet Loss ($L_T$): This contrastive learning component is the "secret sauce" of the framework. The model is presented with an "Anchor" (a real video), a "Negative" (the same video upscaled via SR), and a "Positive" (a different real video). The loss function forces the model to minimize the distance between real representations while maximizing the distance between real and upscaled representations in the feature space.
  3. Variance Loss ($L_V$): To prevent "feature collapse"—a common issue in contrastive learning where the model maps all inputs to a single point—variance loss ensures that the learned features remain diverse and informative.

Data Augmentation Strategy

In most computer vision tasks, aggressive data augmentation (like blurring or noise) improves generalization. However, in forensics, these techniques often destroy the very traces the model is trying to find. The authors discovered that Gaussian blur and JPEG noise significantly degraded performance. Instead, they utilized "Cutout" augmentation—randomly masking parts of the image—which forced the model to look for SR traces across the entire frame without altering the underlying pixel-level artifacts.

Experimental Results

Performance and Generalization

The model was evaluated using several high-quality datasets, including REDS, Vimeo-90K, and the MSU Video Super-Resolution Benchmark. On the REDS and Vimeo test sets, the proposed method achieved near-perfect accuracy (reaching 1.00 on several SR methods), whereas previous state-of-the-art models typically hovered between 0.80 and 0.90.

To test generalization, the model was run against the MSU Benchmark, which contains 32 different upscaling methods. The model successfully identified 30 out of 32 methods, demonstrating that it learns general characteristics of AI-upscaling rather than just memorizing specific artifacts of a single algorithm.

Robustness to Compression

One of the most significant findings was the model's resilience to video compression. Because the hybrid loss function focuses on the nuanced relationship between real and fake textures, the model remained effective even when the forensic traces were partially obscured by H.264 compression. This addresses a major pain point in the field, as most real-world video evidence is compressed.

Conclusion and Future Work

This research demonstrates that combining contrastive learning with supervised classification creates a powerful tool for video forensics. By focusing on temporal traces and using a compression-aware training strategy, the authors have provided a reliable method for verifying video authenticity.

Future work may explore increasing the temporal depth (using 3 or more frames) to see if longer sequences provide more distinct motion traces. Additionally, as generative models like Diffusion-based SR become more prevalent, the model will need to be updated to recognize the specific signatures left by these emerging architectures.

Summary updated

Abstract

Upscaled video detection is a helpful tool in multimedia forensics, but it is a challenging task that involves various upscaling and compression algorithms. There are many resolution-enhancement methods, including interpolation and deep-learning-based super-resolution, and they leave unique traces. In this work, we propose a new upscaled-resolution-detection method based on learning of visual representations using contrastive and cross-entropy losses. To explain how the method detects videos, we systematically review the major components of our framework - in particular, we show that most data-augmentation approaches hinder the learning of the method. Through extensive experiments on various datasets, we demonstrate that our method effectively detects upscaling even in compressed videos and outperforms the state-of-the-art alternatives. The code and models are publicly available at https://github.com/msu-video-group/SRDM
BibTeX
@misc{meshchaninov2022combining,
  author = {Viacheslav Meshchaninov and Ivan Molodetskikh and Dmitriy S. Vatolin},
  title = {{Combining Contrastive and Supervised Learning for Video Super-Resolution Detection}},
  year = {2022},
  eprint = {2205.10406},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2205.10406},
}
Figure