A Comparative Evaluation of Temporal Pooling Methods for Blind Video Quality Assessment

Zhengzhong Tu, Chia-Ju Chen, Li-Heng Chen, Neil Birkbeck, Balu Adsumilli, Alan C. Bovik

Abstract

Temporal pooling is a critical step in Video Quality Assessment (VQA) that aggregates frame-level scores into a single video-level quality metric. This paper presents a comprehensive comparative study of various temporal pooling strategies for blind VQA, particularly focusing on User-Generated Content (UGC). We evaluate classical means, statistical methods, adaptive strategies, and psychovisual models. Furthermore, we introduce EPooling, an ensemble method that fuses multiple pooling strategies using a Support Vector Regressor (SVR). Results on the KoNViD-1k and LIVE-VQC datasets demonstrate that EPooling and psychovisual models like Hysteresis significantly outperform traditional averaging by better aligning with human perceptual biases, such as the "worst-case" effect.

Introduction

Most Video Quality Assessment (VQA) models follow a two-step process: first, they calculate a quality score for every frame using Image Quality Assessment (IQA) algorithms, and then they "pool" these scores over time to reach a final verdict. While this sounds straightforward, User-Generated Content (UGC)—characterized by shaky, poorly lit, and authentically distorted footage—presents a massive challenge. Unlike professional broadcasts, UGC has no "perfect" reference version to compare against. Furthermore, human perception isn't a simple mathematical average; we tend to remember the worst parts of a video more vividly than the mediocre ones. This study seeks to identify which mathematical pooling strategies actually align with how humans judge these messy, real-world videos.

Methods and Pooling Strategies

The authors conducted a large-scale comparative study of several temporal pooling categories:

  • Classical Means: Arithmetic, Harmonic, and Geometric means.
  • Statistical/Minkowski: Minkowski mean and Percentile pooling (focusing on the bottom $k\%$ of scores).
  • Adaptive/Spatial-Temporal: VQPooling, which clusters frames into high and low-quality groups and gives more weight to the lower-quality clusters.
  • Psychovisual/Memory Models: Strategies accounting for the Primacy Effect (remembering the start), the Recency Effect (remembering the end), and Temporal Hysteresis (the tendency for quality ratings to drop sharply during bad patches but recover slowly).

EPooling (Ensemble Pooling)

The authors' novel contribution is EPooling. Instead of relying on a single method, EPooling uses a Support Vector Regressor (SVR) to fuse the predictions of three top-performing, diverse models: the standard Mean, VQPooling, and Hysteresis. This approach aims to capture both general statistical trends and specific human perceptual biases.

Experimental Setup

The study utilized two prominent, large-scale UGC video databases:

  1. KoNViD-1k: 1,200 public-domain videos with diverse authentic distortions.
  2. LIVE-VQC: 585 videos captured by mobile users, featuring complex, commingled distortions.

These datasets are critical because they represent "in-the-wild" video rather than the synthetic, controlled distortions used in older benchmarks. The pooling methods were tested across five different "blind" frame-level quality predictors: NIQE, BRISQUE, GMLOG, HIGRADE, and CORNIA.

Key Results and Discussion

As shown in the performance comparison table, sophisticated pooling methods consistently outperform simple averages, particularly on the more complex LIVE-VQC dataset.

Performance comparison (SRCC/PLCC) of various temporal pooling methods across five blind IQA models on the KoNViD-1k and LIVE-VQC datasets.

Key Insights:

  • The "Worst-Case" Bias: Human judgment is heavily weighted toward the lowest-quality moments. Methods like Percentile pooling and VQPooling, which "punish" the final score based on the worst frames, consistently outperformed simple averages.
  • Hysteresis is Key: The "Temporal Hysteresis" model, which mimics the slow recovery of human satisfaction after a quality drop, was one of the most robust individual pooling methods across different IQA models.
  • EPooling Superiority: The proposed EPooling method achieved the highest or near-highest correlation with human subjective scores (MOS) across almost all tested scenarios. On the LIVE-VQC dataset, employing sophisticated pooling provided a performance boost of approximately ~0.03 in SRCC compared to a simple mean.

Conclusion

No single pooling method is a silver bullet, but by combining statistical averages with psychovisual models, EPooling achieves stable and accurate results across diverse video content. While memory-related effects like Primacy and Recency were less pronounced in the short (8–10 second) clips studied here, they may play a larger role in long-form content. Future research will explore how these strategies scale to longer videos and whether deep learning-based pooling (e.g., LSTMs or Transformers) can more effectively capture the complex "memory" of human visual perception.

Summary updated

Abstract

Many objective video quality assessment (VQA) algorithms include a key step of temporal pooling of frame-level quality scores. However, less attention has been paid to studying the relative efficiencies of different pooling methods on no-reference (blind) VQA. Here we conduct a large-scale comparative evaluation to assess the capabilities and limitations of multiple temporal pooling strategies on blind VQA of user-generated videos. The study yields insights and general guidance regarding the application and selection of temporal pooling models. In addition, we also propose an ensemble pooling model built on top of high-performing temporal pooling models. Our experimental results demonstrate the relative efficacies of the evaluated temporal pooling models, using several popular VQA algorithms, and evaluated on two recent large-scale natural video quality databases. In addition to the new ensemble model, we provide a general recipe for applying temporal pooling of frame-based quality predictions.
BibTeX
@inproceedings{tu2020comparative,
  author = {Zhengzhong Tu and Chia-Ju Chen and Li-Heng Chen and Neil Birkbeck and Balu Adsumilli and Alan C. Bovik},
  title = {{A Comparative Evaluation of Temporal Pooling Methods for Blind Video Quality Assessment}},
  booktitle = {ICIP},
  year = {2020},
  doi = {10.1109/icip40778.2020.9191169},
}
Figure