Abstract
High frame rate (HFR) videos are becoming increasingly common with the tremendous popularity of live, high-action streaming content such as sports. Although HFR contents are generally of very high quality, high bandwidth requirements make them challenging to deliver efficiently, while simultaneously maintaining their quality. To optimize trade-offs between bandwidth requirements and video quality, in terms of frame rate adaptation, it is imperative to understand the intricate relationship between frame rate and perceptual video quality. Towards advancing progression in this direction we designed a new subjective resource, called the LIVE-YouTube-HFR (LIVE-YT-HFR) dataset, which is comprised of 480 videos having 6 different frame rates, obtained from 16 diverse contents. In order to understand the combined effects of compression and frame rate adjustment, we also processed videos at 5 compression levels at each frame rate. To obtain subjective labels on the videos, we conducted a human study yielding 19,000 human quality ratings obtained from a pool of 85 human subjects. We also conducted a holistic evaluation of existing state-of-the-art Full and No-Reference video quality algorithms, and statistically benchmarked their performance on the new database. The LIVE-YT-HFR database has been made available online for public use and evaluation purposes.
Introduction
As streaming services pivot toward high-action content like sports and gaming, High Frame Rate (HFR) video—defined as anything above the traditional 24–30 fps—is becoming the new standard. However, HFR comes with a massive bandwidth penalty. To deliver video efficiently, providers need to identify the "sweet spot" where increasing the frame rate ceases to provide a noticeable benefit to the viewer. Existing Video Quality Assessment (VQA) research has largely focused on spatial resolution (4K/8K) or dynamic range (HDR), leaving a gap in our understanding of the interplay between frame rate, compression artifacts, and human perception.
Methodology
The study combined a massive subjective human study with a rigorous benchmark of automated "objective" algorithms.
To create different frame rates from 120 fps source material, the researchers used frame dropping rather than frame averaging. This method avoids artificial motion blur and better simulates how cameras natively capture lower frame rates. For the subjective study, they employed the Single-Stimulus Continuous Quality Evaluation (SSCQE) procedure, where 85 subjects provided over 19,000 ratings on a continuous scale from "Bad" to "Excellent."
Finally, the team tested several Full-Reference (FR) and No-Reference (NR) algorithms, including industry standards like VMAF and PSNR, to determine how well they correlate with human Mean Opinion Scores (MOS).
The LIVE-YouTube-HFR Dataset
The researchers introduced the LIVE-YouTube-HFR (LIVE-YT-HFR) dataset, which is significantly larger and more diverse in its frame-rate sampling than previous benchmarks.
The dataset includes:
- Content: 480 videos derived from 16 diverse source contents, ranging from sports and nature to indoor activities.
- Frame Rates: Six levels (24, 30, 60, 82, 98, and 120 fps). The 82 and 98 fps rates were included to provide fine-grained data in the high-frequency range.
- Compression: Five levels of VP9 compression (from lossless to high distortion).
- Resolution: A mix of 1080p and 4K (UHD-1).
Perceptual Insights
- The 60 fps Plateau: Increasing frame rate leads to higher perceived quality, but there are diminishing returns beyond 60 fps. While 120 fps is objectively better, the jump from 30 to 60 is much more perceptible to a viewer than the jump from 60 to 120.
- The "Judder" Factor: Camera motion is a critical variable. Videos with significant camera movement suffer much more at low frame rates due to "judder" or strobing artifacts. For these videos, HFR provides a much more dramatic improvement in perceived quality.
- Algorithmic Spatial Bias: Standard metrics like PSNR and SSIM are "temporally blind." They may assign a high score to a 24 fps video because individual frames look sharp, ignoring the choppiness perceived by a human viewer.
Performance Benchmarking
The study achieved an extremely high inter-subject correlation (0.96 SROCC), proving that human perception of frame-rate quality is remarkably consistent.
Among Full-Reference models, GSTI (SROCC: 0.7909) and VMAF (SROCC: 0.7303) outperformed traditional metrics like PSNR (0.6950) and SSIM (0.4494). These models likely succeed because they incorporate temporal features, such as frame-differencing. Conversely, No-Reference models struggled significantly; the best model (V-BLIINDS) reached an SROCC of only 0.430, highlighting a major need for better "blind" VQA tools for HFR content.
Conclusion and Future Work
While the LIVE-YT-HFR dataset provides a vital baseline, several avenues for future research remain. The current study focused exclusively on the VP9 codec and frame-dropping downsampling. Future work could explore how newer codecs like AV1 or VVC interact with HFR, and how complex frame-rate conversion (interpolation or averaging) introduces different artifacts. Additionally, as 240Hz or 360Hz displays become common in gaming, the "plateau" of diminishing returns may shift further upward.
Summary updated