Abstract
Near-duplicate video detection (NDVD) is critical for copyright enforcement and content management, yet traditional fingerprinting methods often fail under combined video modifications. This paper proposes a novel framework that utilizes a high-order tensor model (Tucker decomposition) to fuse global, local, and temporal features. By mining the "consensus" between diverse feature types rather than relying on simple concatenation, the system generates a "comprehensive feature" that is highly resilient to complex attacks. Experimental results on the CC_WEB_VIDEO and OV datasets demonstrate superior robustness, particularly against combined modifications, while maintaining computational efficiency through an auxiliary matching strategy.
Introduction
The explosion of online video content has made near-duplicate video detection (NDVD) a significant challenge for copyright holders and search engines. Digital fingerprinting—creating a compact digital digest of video content—is the standard approach for identifying duplicates. However, while current methods handle single modifications (like noise or blurring) effectively, they often struggle with "combined modifications" such as simultaneous cropping, brightness changes, and noise. This paper addresses this gap by moving beyond single-feature reliance, proposing a tensor-based model to integrate multiple feature types into a single, robust signature.
Methodology
The proposed framework consists of three primary stages: tensor construction, decomposition, and fingerprint generation.
1. Video Tensor Construction
A video is represented as a 3rd-order tensor. Unlike flat vectors, this 3D data block captures the complex relationships between different feature types, their specific values, and the temporal sequence of the video.
2. Tensor Decomposition (Tucker Model)
The large video tensor is decomposed using the Tucker model into a small "core tensor" and three "factor matrices." To avoid manual parameter tuning, the authors employ Automatic Relevance Determination (ARD), a Bayesian approach that automatically identifies the optimal number of components for decomposition, ensuring the model captures only the most relevant information.
3. Fingerprint Generation and Matching
The final fingerprint is derived from the factor matrices, representing the intrinsic "comprehensive feature" of the video. For large-scale searches, the core tensor acts as a rapid "match tag" to filter out obvious non-matches before a detailed comparison is performed, significantly boosting search efficiency.
The Power of Consensus
A core innovation of this work is the shift from feature concatenation to feature consensus. Simple concatenation often ignores the underlying correlations between different "views" (e.g., color vs. texture) of the same video.
By using tensor decomposition, the model identifies the mathematical consensus among these diverse features, effectively filtering out noise and irrelevant information specific to any single feature type. This process is mathematically "well-conditioned," meaning small changes in the input (video attacks) result in only minimal changes in the output fingerprint.
Experimental Results
The method was rigorously tested against state-of-the-art techniques (LRTA, CGO, 3D-DCT) using the CC_WEB_VIDEO and OV datasets. The researchers applied 11 single-mode attacks and 4 complex combined modifications to nearly 20,000 videos.
- Superior Robustness: The proposed method consistently achieved higher F-scores. For example, under "Logo Insertion," it scored 0.9476 compared to LRTA's 0.8908.
- Combined Attack Performance: In a complex combined attack involving cropping, flipping, and picture-in-picture effects, the proposed method maintained an F-score of 0.8725, significantly outperforming 3D-DCT (0.5745).
- Efficiency: The auxiliary matching strategy using the core tensor proved highly effective for large-scale database operations, reducing search time without sacrificing accuracy.
Conclusion
This research demonstrates that high-order tensor models provide a powerful mathematical framework for fusing diverse video features. By mining the consensus between features, the resulting fingerprints achieve unprecedented robustness against combined video attacks. Future work will focus on automated feature selection to further optimize the input for the tensor model across different video genres.
Summary updated