Abstract
As Super-Resolution (SR) technology advances, the ability to distinguish between native-resolution video and artificially upscaled content has become a critical forensic challenge. Modern AI-based SR methods leave subtle traces that are often obliterated by standard video compression. This paper presents a robust forensic framework that combines contrastive and supervised learning to detect these "fake" high-resolution videos. By utilizing a hybrid loss function and a temporal-aware architecture, the proposed model maintains high detection accuracy even in heavily compressed scenarios, significantly outperforming previous state-of-the-art methods.
Introduction
The proliferation of high-definition displays has driven a massive demand for 4K and 8K content. To meet this demand, Super-Resolution (SR) techniques—particularly those powered by deep learning—are frequently used to upscale lower-resolution footage. While beneficial for consumer entertainment, this capability creates a forensic vulnerability. Dishonest vendors can misrepresent upscaled content as native high-resolution footage, leading to legal and ethical concerns in stock photography, evidence authentication, and consumer electronics marketing.
Traditional forensic methods typically look for mathematical artifacts left by simple interpolation (e.g., bicubic upscaling). However, modern neural-network-based SR methods produce much more sophisticated textures. Furthermore, the standard pipeline for video distribution involves heavy compression (such as H.264), which acts as a "forensic bleach," removing the high-frequency micro-traces that standard algorithms rely on for detection.
Methodology
Architectural Design
The researchers developed a deep-learning model based on a ResNet-50 feature encoder. Recognizing that SR artifacts often manifest as temporal inconsistencies between frames, the authors modified the first layer of the network. Instead of the standard 3-channel (RGB) input for a single image, the model accepts two consecutive frames (6 channels). This allows the network to capture motion-related traces and temporal artifacts that are absent in native-resolution sequences.
The Hybrid Loss Function
The core innovation of this research lies in its training strategy, which employs three distinct loss functions to ensure the model learns robust, discriminative features:
- Cross-Entropy Loss ($L_{CE}$): This provides the standard supervised signal, training the model to classify inputs into "Real" (native) or "Fake" (SR-upscaled) categories.
- Triplet Loss ($L_T$): This contrastive learning component is the "secret sauce" of the framework. The model is presented with an "Anchor" (a real video), a "Negative" (the same video upscaled via SR), and a "Positive" (a different real video). The loss function forces the model to minimize the distance between real representations while maximizing the distance between real and upscaled representations in the feature space.
- Variance Loss ($L_V$): To prevent "feature collapse"—a common issue in contrastive learning where the model maps all inputs to a single point—variance loss ensures that the learned features remain diverse and informative.
Data Augmentation Strategy
In most computer vision tasks, aggressive data augmentation (like blurring or noise) improves generalization. However, in forensics, these techniques often destroy the very traces the model is trying to find. The authors discovered that Gaussian blur and JPEG noise significantly degraded performance. Instead, they utilized "Cutout" augmentation—randomly masking parts of the image—which forced the model to look for SR traces across the entire frame without altering the underlying pixel-level artifacts.
Experimental Results
Performance and Generalization
The model was evaluated using several high-quality datasets, including REDS, Vimeo-90K, and the MSU Video Super-Resolution Benchmark. On the REDS and Vimeo test sets, the proposed method achieved near-perfect accuracy (reaching 1.00 on several SR methods), whereas previous state-of-the-art models typically hovered between 0.80 and 0.90.
To test generalization, the model was run against the MSU Benchmark, which contains 32 different upscaling methods. The model successfully identified 30 out of 32 methods, demonstrating that it learns general characteristics of AI-upscaling rather than just memorizing specific artifacts of a single algorithm.
Robustness to Compression
One of the most significant findings was the model's resilience to video compression. Because the hybrid loss function focuses on the nuanced relationship between real and fake textures, the model remained effective even when the forensic traces were partially obscured by H.264 compression. This addresses a major pain point in the field, as most real-world video evidence is compressed.
Conclusion and Future Work
This research demonstrates that combining contrastive learning with supervised classification creates a powerful tool for video forensics. By focusing on temporal traces and using a compression-aware training strategy, the authors have provided a reliable method for verifying video authenticity.
Future work may explore increasing the temporal depth (using 3 or more frames) to see if longer sequences provide more distinct motion traces. Additionally, as generative models like Diffusion-based SR become more prevalent, the model will need to be updated to recognize the specific signatures left by these emerging architectures.
Summary updated