Abstract
Traditional Video Quality Assessment (VQA) methods often struggle with "in-the-wild" content due to a reliance on reference videos or extensive training on human-labeled datasets. This paper explores a bio-inspired approach to Blind VQA (BVQA) through the development of the Temporal Perceptual Quality Index (TPQI). By leveraging the hypothesis that the human visual system (HVS) "straightens" temporal trajectories of natural signals, TPQI measures video quality based on the geometric regularity of perceptual representations. Experimental results across four major datasets demonstrate that this completely blind, training-free metric achieves state-of-the-art performance, occasionally outperforming models trained on human opinions.
Introduction
Assessing the quality of user-generated content (UGC) is a significant challenge in computer vision. Unlike controlled laboratory videos, UGC often contains complex, authentic distortions such as camera shake, poor lighting, and compression artifacts. Most existing VQA tools are "opinion-aware," requiring massive datasets of human-rated videos to learn quality features. Furthermore, many models prioritize spatial sharpness over temporal consistency, failing to capture the nuances of motion perception. This research introduces a "completely blind" method that requires neither a reference video nor prior training, instead relying on the biological principles of human vision.
Methodology: The Temporal Perceptual Quality Index (TPQI)
The TPQI framework is built upon the neuroscience hypothesis that the HVS transforms complex visual inputs into more regular, linear representations as they progress from the retina to the primary visual cortex. The method operates through a multi-stage pipeline:
1. Perceptual Transformation
Input video frames are processed through computational models of the Lateral Geniculate Nucleus (LGN) and the Primary Visual Cortex (V1). These models utilize Difference of Gaussian (DoG) and Gabor filters to simulate the neural responses of the human brain to visual stimuli.
2. Trajectory Mapping and the VPT Descriptor
To manage the high dimensionality of neural responses, Principal Component Analysis (PCA) is used to project these responses into a low-dimensional space, creating a "temporal trajectory." The core of the TPQI is the Video Perceptual Trajectory (VPT) descriptor, which analyzes two geometric properties:
- Curvature (Direction Change): Measures the smoothness of the trajectory. High-quality videos typically exhibit straighter paths, while distortions introduce jaggedness or high curvature.
- Compactness (Distance Change): Measures the displacement between frames in the perceptual space. This captures the intensity of distortions that might not significantly alter the path's direction but affect its temporal stability.
3. Metric Fusion
To provide a comprehensive assessment, the temporal scores derived from the VPT descriptor are fused with the Naturalness Image Quality Evaluator (NIQE), a well-established spatial quality metric. This combination allows the model to account for both frame-level clarity and temporal fluidity.
Experimental Setup
The effectiveness of TPQI was evaluated on four prominent "in-the-wild" datasets:
- KoNViD-1k: 1,200 public-domain videos.
- LIVE-VQC: 585 videos featuring diverse motion and complex distortions.
- CVD2014: 234 videos captured by 78 different mobile devices.
- YouTube-UGC: Over 1,100 clips spanning various categories, including gaming and sports.
Results and Discussion
TPQI demonstrated competitive performance against both blind and opinion-aware metrics. On the KoNViD-1k and LIVE-VQC datasets, the combination of TPQI and NIQE achieved Spearman’s Rank Correlation Coefficients (SRCC) of 0.693 and 0.718, respectively.
Key Insights
- The Straightness Hypothesis: The research confirms that the geometric "straightness" of a video's perceptual trajectory is a powerful proxy for human-perceived quality. Natural, undistorted motion results in linear trajectories, while artifacts cause measurable geometric deviations.
- Biological vs. Deep Learning Models: Interestingly, the bio-inspired LGN/V1 features outperformed features extracted from deep learning architectures like ResNet or VGG for this specific task. This suggests that fundamental biological rules of perception may be more efficient for quality assessment than generalized deep features.
- Zero-Shot Capability: As a completely blind metric, TPQI requires no parameter tuning or training, making it immediately applicable to new datasets and real-world streaming environments.
Limitations and Future Work
While TPQI excels with natural scenes, its performance declined on the YouTube-UGC dataset. This is attributed to the presence of "unnatural" content, such as animations and screen recordings, which do not adhere to the evolutionary "straightness" rules of natural visual processing. Future research will focus on adapting perceptual trajectories for non-natural content and exploring applications in video restoration and compression optimization.
Conclusion
TPQI represents a significant step forward in training-free video quality assessment. By mimicking the HVS's tendency to linearize visual signals, the model provides a robust, mathematically grounded approach to measuring quality without the need for human-labeled data. Its success highlights the continued relevance of bio-inspired models in an era dominated by black-box artificial intelligence.
Summary updated