TL;DR
This paper introduces a new dataset and an improved video quality prediction model (VMAFc) that explicitly accounts for chroma distortions, which are often overlooked by existing models. By considering both luma and chroma quality, they demonstrate potential for significant bitrate savings in video compression without sacrificing perceived quality.
What Problem Does This Paper Address?
Measuring and predicting the perceived quality of digital videos is crucial for applications like streaming and video conferencing. While many objective video quality assessment (VQA) models exist, they predominantly focus on the luminance (brightness) component of video, largely ignoring the color (chroma) information. This is problematic because chroma distortions, such as color bleeding or desaturation, can significantly impact perceived quality, and current models fail to capture these effects accurately. Furthermore, the decoupling of luma and chroma compression in modern codecs presents an opportunity for bitrate reduction that is missed by luma-only VQA models. The paper aims to bridge this gap by investigating the perceptual impact of chroma distortions and developing VQA models that incorporate this information.
Methods
The core of this work involves a two-pronged approach:
- Subjective Study and Dataset Creation: The authors designed and conducted a subjective experiment with 34 participants to gather perceptual quality scores for videos with varying levels of luma and chroma distortions. They introduced these distortions by manipulating quantization parameters in the HEVC video codec. Crucially, they developed a method to generate videos where luma and chroma distortions could be independently controlled, moving beyond the typical coupling in standard codecs. This resulted in the creation of the NFLX-Color (NFLXc) dataset, a benchmark specifically designed to study chroma distortions.
- Objective VQA Model Development: Leveraging the insights from their subjective study and the NFLXc dataset, the authors proposed an enhanced VQA model, termed VMAFc. This model builds upon the existing VMAF framework by incorporating new features derived from the chroma channels. They explored various chromatic features and found that multiscale Additive Distortion Metric (ADM) features performed particularly well. These features were then integrated into a Support Vector Regressor (SVR) to create the VMAFc model.
Datasets
- NFLX-Color (NFLXc) Dataset: This is the primary new dataset introduced in the paper. It comprises 210 distorted videos derived from 21 pristine Full High Definition (FHD) source sequences. The distortions were generated by varying luma and chroma quantization parameters, allowing for independent control and a wide range of quality levels.
- Existing VQA Datasets: For evaluation and comparison, the authors utilized several publicly available VQA datasets, including LIVE-VQA, LIVE-MBL, CSIQ-VQA, BVI-HD, VQEG, SHVC, EPFL, and VMAF+.
Key Insights and Ideas
- Chroma Matters: Existing VQA models that ignore chroma information are fundamentally limited and can lead to inaccurate quality predictions, especially when chroma distortions are present.
- Bitrate Savings Potential: By decoupling luma and chroma compression, there's a significant opportunity to reduce bitrate consumption. The paper demonstrates that aggressive chroma quantization can yield substantial bitrate reductions with minimal perceived quality loss, particularly at higher bitrates.
- Multiscale Chroma Features: Human visual perception of color is more sensitive to lower frequencies. Multiscale ADM features, which capture these lower-frequency chromatic distortions, proved highly effective for predicting perceived quality.
- Monotonicity is Key: An ideal VQA model should exhibit monotonicity – as distortion increases (e.g., chroma QP offset increases), the predicted quality score should decrease or stay the same. The proposed VMAFc model demonstrates this desirable property, unlike some baseline models.
Key Results
- The newly developed VMAFc model achieved state-of-the-art performance on the NFLXc dataset, outperforming existing VQA models, including VMAF 0.6.1, in predicting subjective chroma distortion quality.
- The paper demonstrates that by increasing the chroma QP offset, bitrate reductions of up to 36% were achieved in specific examples without significant perceived quality degradation.
- The VMAFc model showed improved performance across various datasets compared to VMAF 0.6.1, particularly on datasets where chroma distortions were more prominent.
- Statistical significance tests confirmed that VMAFc and iCID statistically surpassed most other models, with VMAFc showing strong overall performance.
Limitations and Future Work
- Dataset Limitations: The NFLXc dataset, while comprehensive for chroma distortions, is still limited by the number of subjects and distortion types. It does not include distortions like scaling artifacts or simulated transmission errors, which are common in real-world scenarios.
- Real-world Compression: The study focused on HEVC-compressed videos. Future work could explore other codecs and compression techniques.
- Feature Exploration: While promising, the authors suggest further research into new chromatic features and more sophisticated machine learning models for even better chroma distortion prediction.
- Joint Optimization: The ultimate goal is to use these insights to jointly optimize luma and chroma compression in video encoders, leading to more efficient and perceptually pleasing video delivery. The paper lays the groundwork for this by highlighting the potential for bitrate savings through creative chroma compression.
Summary updated