TL;DR
This paper introduces a novel "eigen GRD" model to accurately predict video quality based on encoding parameters like bitrate and spatial resolution. By analyzing the mathematical properties of generalized rate-distortion (GRD) functions and leveraging a large dataset of real-world GRD curves, they demonstrate that these functions reside in a low-dimensional subspace, enabling efficient and accurate estimation from sparse data.
What Problem Does This Paper Address?
Video compression is a complex process where the trade-off between file size (bitrate) and visual quality is crucial. Traditional Rate-Distortion (RD) theory focuses on a single dimension (bitrate vs. distortion). However, modern video delivery involves multiple encoding parameters, such as spatial resolution, frame rate, and bit depth, all of which influence the final perceived video quality. This "generalized RD" (GRD) relationship is content-dependent and difficult to model accurately, especially when only a few data points (e.g., from limited encoding tests) are available. Existing methods often make strong assumptions or require dense sampling, which is computationally expensive. This paper aims to bridge this gap by providing a theoretically sound and computationally efficient model for estimating GRD functions from sparse measurements.
Methods
The core of the paper lies in developing a mathematical framework for GRD functions.
- Theoretical Space (W): They define a theoretical functional space 'W' for GRD functions, showing it's a convex set within a Hilbert space. This mathematical grounding is key to ensuring the validity of their model.
- Low-Dimensional Subspace: They hypothesize and then demonstrate that real-world GRD functions occupy a low-dimensional subspace within 'W'. This is a crucial insight, as it implies that a few parameters can capture most of the variation.
- Eigen GRD Model: Leveraging the low-dimensionality, they use Principal Component Analysis (PCA) on a large dataset of GRD functions to derive an "eigen basis." This eigen basis, along with the mean GRD function, forms their "eigen GRD" (eGRD) model.
- Sparse Estimation: The eGRD model allows for efficient estimation of GRD functions from a small number of "attribute-quality pairs" (e.g., a few encoded video versions and their measured quality). This is framed as a projection-onto-convex-sets (POCS) problem, solved using quadratic programming.
- Monotonicity Constraints: They incorporate axial monotonicity constraints (e.g., higher bitrate generally means better quality at a given resolution) into their model, which helps regularize the estimation and prevent overfitting, especially with sparse data.
Datasets
- Waterloo GRD Database: The authors introduce a large-scale database of GRD functions. This database was constructed by:
- Collecting 1,000 high-quality videos.
- Downsampling them to six spatial resolutions.
- Encoding these at various bitrates (100 kbps to 9 Mbps) using H.264 and VP9 encoders.
- Measuring perceptual quality using SSIMplus (a state-of-the-art objective VQA metric). This resulted in approximately 1.08 million video representations. They then processed this data to obtain 2,000 GRD functions (from 1,000 videos and 2 encoders).
Key Insights and Ideas
- GRD Functions are Low-Dimensional: The most significant insight is that real-world GRD functions, despite their complexity, can be effectively represented by a small number of "eigen" components. This drastically reduces the data needed for accurate modeling.
- Mathematical Foundation Matters: By defining a theoretical space 'W' and proving its convexity, the authors provide a strong theoretical basis for their computational model, ensuring its validity.
- Monotonicity is Key for Sparsity: Incorporating monotonicity constraints is crucial for achieving accurate GRD estimation from sparse data, preventing overfitting and ensuring physically plausible results.
- Eigen Basis Outperforms Fixed Bases: The learned eigen basis derived from data is significantly more effective at representing GRD functions than fixed bases like polynomials or trigonometric functions, especially with limited training data.
Key Results
- High Accuracy with Few Samples: The eGRD model can accurately estimate GRD functions with as few as 8 samples, significantly outperforming existing methods like reciprocal and logarithmic regression.
- Superior Performance: Experiments show that eGRD achieves significantly lower Root Mean Square Error (RMSE) and $l_\infty$ error compared to polynomial and trigonometric GRD models, especially when using a small number of basis functions. For instance, 2 eigenvectors can outperform 20 polynomial or trigonometric vectors.
- Robustness: The model demonstrates robustness across different VQA metrics (SSIMplus and VMAF) and even when trained on one encoder (H.264 or VP9) and tested on another, indicating its generalizability.
- Video Codec Comparison: The eGRD model significantly improves the accuracy of video codec comparison, outperforming traditional methods like BD and PCHIP, particularly in estimating quality gains and bitrate savings.
Limitations and Future Work
- Encoder Dependency: While the model shows good generalization, training on one encoder and testing on another leads to a slight decline in accuracy. The authors suggest retraining eGRD on the new encoder for optimal performance with substantially different codecs.
- VQA Metric Dependency: Although the paper demonstrates generality with VMAF, the underlying assumptions of GRD functions might still be influenced by the choice of VQA metric.
- Future Work: The authors suggest that the low-dimensional nature of GRD functions could facilitate reduced-reference VQA, where only a few coefficients need to be transmitted. They also highlight the importance of monotonicity for practical applications and future video codec comparisons.
Summary updated