Most video quality research happens in a "perfect" vacuum: a high-end monitor, a dark room, and a viewer sitting at exactly the right distance. But in reality, users watch Netflix on dim tablets in bed or 4K TVs from a distant couch. In these scenarios, the compression artifacts that look glaring in a lab often become invisible.
TL;DR
This paper introduces HDR-VDC, a new 4K HDR dataset that measures how viewing distance and display brightness change our perception of video quality. The authors demonstrate that by using a simple pre-processing pipeline—accounting for display physics and human vision—standard quality metrics can be "fixed" to predict real-world performance, potentially saving up to 80% in bitrate for distant viewers.
What Problem Does This Paper Address?
Standard Image and Video Quality Metrics (IQMs/VQMs) like PSNR or VMAF typically assume fixed, optimal viewing conditions. They are "blind" to the fact that human visual acuity drops as you move away from a screen and that our sensitivity to contrast changes with display brightness.
This creates a massive inefficiency in video streaming. If a streaming service knows you are watching a 4K screen from 10 feet away, it could send a much lower bitrate stream that looks identical to the high-bitrate version at that distance. To enable this "viewing-condition-aware" streaming, we need metrics that can accurately predict quality across a spectrum of real-world environments.
Methods
The authors tackled this by creating a robust subjective study and a metric adaptation framework.
1. The Subjective Study
Instead of asking users to rate videos on a scale of 1-10 (which is noisy), they used a Pairwise Comparison (PC) protocol with Active Sampling (ASAP). Users saw two versions of a video and simply picked which one looked closer to the original. This is a much more sensitive way to measure subtle perceptual differences.
2. The Adaptation Pipeline
To make existing metrics "aware" of viewing conditions, the authors developed a three-step pre-processing framework:
- Display Model: Maps the digital video values to the actual light (luminance) emitted by a specific screen (in this case, an LG OLED G2).
- PU21 Encoding: Converts luminance into a "perceptually uniform" space. This accounts for the fact that our eyes are more sensitive to changes in dark areas than in very bright areas.
- Perspective Projection Scaling (PPS): Rescales the video frames based on the viewer's distance. If a viewer is twice as far away, the image is downsampled to simulate the lower resolution projected onto the retina.
Datasets
The core of the work is the HDR-VDC (High Dynamic Range - Viewing Distance and Conditions) dataset:
- Content: 16 HDR source sequences (12 at 4K, 4 at FHD) including natural scenes, animation, and gaming.
- Distortions: AV1 compression and Lanczos upscaling.
- Conditions: Tested at two viewing distances (60 and 120 pixels per degree) and two luminance levels (bright vs. dimmed by a factor of 8).
- Availability: The dataset and code are publicly available, providing a much-needed benchmark for HDR quality research.
Key Insights and Ideas
- Distance > Brightness: Viewing distance has a "medium" effect on perceived quality, while display luminance has a "small" effect. Basically, moving your chair back hides artifacts much better than turning down your screen brightness.
- The "Aha" Moment for Bitrate: The study found that for a viewer at a "far" distance (120 ppd), you could achieve the same perceived quality with an eightfold decrease in bitrate compared to a "near" viewer.
- General Quality vs. Condition Sensitivity: A metric can be great at ranking general video quality but terrible at predicting how quality changes when you move away from the screen. The authors "disentangled" these two properties to show which metrics are truly robust.
Key Results
- Top Performers: When adapted with the proposed pipeline, VMAF-4K, VMAF, HaarPSI, and ColorVideoVDP emerged as the most reliable metrics.
- The Failure of Classics: Widely used "simple" metrics like PSNR, SSIM, and sCIELab performed very poorly, even with adaptation. They simply don't capture the nuances of HDR and human contrast sensitivity.
- Native vs. Adapted: Metrics that natively model the human Contrast Sensitivity Function (CSF)—like HDR-VDP-3 and FovVideoVDP—performed well without needing the external PPS scaling, validating their internal biological models.
Limitations and Future Work
- Limited Scope: The study only used two distances and two brightness levels on a single OLED display. Results might vary on mobile LCDs or in high-glare environments.
- Distortion Types: The dataset focuses on AV1 compression and upscaling. It doesn't yet cover other streaming "pain points" like stalling, frame drops, or temporal jitter.
- Client-Side Estimation: For this to work in the real world, the streaming app needs to know how far away you are. Future work involves using device sensors (like front-facing cameras or ambient light sensors) to estimate these conditions automatically.
For those interested in the math or downloading the HDR-VDC dataset, check out the project's GitHub repository.
Summary updated