Does the Timing of Quality Drops Matter? Volatility-Clustering Statistics for Video Quality

A video quality metric usually scores each frame and then averages the scores. Two clips can have the same average and the same spread of per-frame quality and still look very different to a viewer. In one, bad frames are scattered as rapid flicker. In the other, they bunch into a single sustained breakdown. Finance has a mature toolkit for exactly this distinction. Researchers there study volatility clustering: the observation that large price moves follow large moves, so turbulence comes in bursts whose memory can decay slowly, as a power law. This proposal treats the per-frame quality of a video as a price series. It asks whether human judgments depend on how quality drops cluster in time, holding fixed how often they occur. If they do, it asks whether a few clustering statistics borrowed from econometrics close part of the gap between objective scores and human opinion.

The opportunity is concrete. Advancing Video Quality Assessment for AIGC observed that AI-generated videos swing between photorealistic and broken frames. It argued that averages are "unfair" because viewers punish flicker between great and terrible more than steady mediocrity. Its fix is a loss over the distribution of frame scores, and that loss ignores their order. No one has tested, with order manipulated and everything else held fixed, whether ordering carries perceptual weight. No one has tested whether the right description is short memory, long memory, or simply a few regime breaks.

Core Claim

We claim that, at a fixed marginal distribution of per-frame quality, human mean opinion scores (MOS, the average rating a panel of viewers gives a clip) depend on the temporal persistence of the quality-change series. We further claim this dependence is not monotone. Anti-persistent alternation (flicker) and highly persistent breakdown episodes should both be penalized relative to an intermediate structure, and a scalar persistence measure plus an episode-duration measure should capture most of the effect.

Let $q_t$ be the frame-level quality score at frame $t$ and $v_t = |q_t - q_{t-1}|$ its absolute change, the analogue of absolute returns. Persistence is summarized by the Hurst exponent $H$ of $v_t$. The value $H = 0.5$ means no memory, $H > 0.5$ means long memory, and $H < 0.5$ means alternation. The prediction under test is that adding $H$ and the mean length of low-quality runs to a pooled model of mean and standard deviation improves MOS prediction. It must do so on real data and on a controlled stimulus set where only ordering varies.

The new element is the matched-marginal test. Existing work changes what the drops look like or how many there are. We change only when they happen.

Research Program

The program has three parts. The first is a controlled perceptual study that establishes whether ordering matters at all. Without that result, everything downstream is curve fitting.

We take high-MOS source clips of 10–20 seconds (300–600 frames) from YouTube-UGC and LSVQ. We inject one distortion family at a time: compression blocking, blur, or a generative-style warp. The frame schedule is a binary or graded "bad frame" sequence drawn from processes with identical marginal rates and different temporal structure. The processes are i.i.d. Bernoulli, a two-state Markov chain (short memory, geometric run lengths), and fractional-Gaussian-thresholded schedules spanning $H \in \{0.3, 0.5, 0.7, 0.9\}$. Two further schedules borrow from microscopic market models. One is a single contiguous block. The other is a threshold-crossing schedule in the spirit of On a universal mechanism for long ranged volatility correlations, where agents switch state when a random-walk score crosses zero. That mechanism yields heavy-tailed $t^{-3/2}$ dwell times. Here it supplies a natural generator of bursty schedules with no characteristic duration. Viewers rate every schedule of a given source and distortion. The design is within-source, so content cannot explain differences.

Clip length is chosen for statistical reasons, not convenience. Bootstrap-Based Test for Volatility Shifts in GARCH against Long-Range Dependence found that even at $n=500$ asymptotic tests misbehave under heavy tails. Its empirical size was 0.01–0.02 against a nominal 0.05, while its bootstrap held near 0.05. Persistence cannot be measured on 16-frame clips. Our observational analysis is limited accordingly.

The second part builds the estimator and the pooled score. Frame scores come from an existing frame-level metric. That metric is full-reference BiRQA: Bidirectional Robust Quality Assessment for Images when a reference exists. Otherwise it is the frame branch of a no-reference video model. These scores are noisy estimates of latent quality.

This noise matters for the estimator choice. Estimating Long Memory in Volatility shows that standard log-periodogram estimators are badly biased when a long-memory signal carries additive noise. Its modified local Whittle estimator, which adds an explicit white-noise term, is the method we reuse for $H$. We will compare it with detrended fluctuation analysis, the estimator used in Modeling and estimation on long memory stochastic volatility for index prices of FTSE Bursa Malaysia KLCI. That paper found $H \approx 0.5$ for returns and $H \approx 0.67$–$0.76$ for volatility proxies. It is the template for showing that a magnitude series carries memory while the signed series does not.

The pooled model is deliberately small:

$$ \widehat{\text{MOS}} = a + b\,\bar q + c\,s_q + \gamma_1 \hat H + \gamma_2 (\hat H - 0.5)^2 + \gamma_3 \bar L, $$

Here $\bar q$ and $s_q$ are the mean and standard deviation of $q_t$, $\hat H$ is the estimated Hurst exponent of $v_t$, and $\bar L$ is the mean length of runs below a quality threshold. The quadratic term carries the non-monotone hypothesis.

The third part handles a confound. Scene cuts produce single huge values of $v_t$ that can make a series look long-memory, which is the video version of the structural-break problem. We apply a residual-bootstrap CUSUM change-point test, following Bootstrap-Based Test for Volatility Shifts in GARCH against Long-Range Dependence, to locate breaks. We then estimate persistence within segments. On generated video, we ask whether the detected breaks are real cuts or generative collapses.

Baselines

The naive reference is mean pooling of the same frame-level scores used by our statistics. Any gain must come from temporal structure, not a better frame scorer.

A stronger hand-crafted reference adds the standard deviation, the minimum, and the 10th percentile of $q_t$. These capture "how bad, how often" without order. They are the honest version of "just add variance."

The strongest published competitors differ by regime. On AI-generated video, the strongest is the model of Advancing Video Quality Assessment for AIGC, trained with its frame consistency loss. It reports PLCC 0.757 on T2VQA-DB, 3.1% above SAMA, and its full model has an SROCC of 0.742. On user-generated video, the strongest is DIVA-VQA, which works from one-step frame differences. Its DIVA-VQA: Detecting Inter-Frame Variations in UGC Video Quality reports rank correlation of 0.898 (DIVA-VQA-L) and 0.886 (DIVA-VQA-B) on KoNViD-1k. For fairness, we will append our statistics to each model's own frame-level outputs and refit only the pooling head. We will use the same train/test splits, the same number of hyperparameter trials per method, and identical compute. Beating these means a statistically significant gain in SROCC and PLCC across repeated splits, not one lucky split.

The most dangerous competitor is a learned temporal pooler: a small GRU or transformer over the sequence $q_t$, trained on MOS with the same budget. No reported result was found on Lacuna for a learned pooler on these frame-score sequences, so we will train it ourselves.

Two ablations isolate the key ingredient. In the first, each clip's frames are randomly shuffled before computing statistics. This destroys order but keeps the marginal distribution, so any gain that survives shuffling is not about clustering. In the second, $\hat H$ is replaced with GARCH(1,1) persistence ($\alpha + \beta$), a short-memory measure whose influence decays exponentially, as Park notes. This tests whether long memory is needed or whether short-memory clustering suffices.

Why not just…?

Why not just train a sequence model on frame scores and let it learn the clustering? This is the real threat. With enough MOS-labeled data, a GRU could learn any order-dependent rule. Our answer has two parts. First, the controlled study is a scientific question a black-box pooler cannot answer: whether humans respond to ordering, and in which direction. Second, the AIGC and UGC datasets hold hundreds to low thousands of clips, where a five-parameter model with the right inductive bias may beat a learned one. We will run the learned pooler at matched budget, then train it on the controlled set and test it on held-out schedule types, such as an $H$ value it never saw. If it matches our statistics in and out of distribution, we will report the statistics as an interpretable description, not a better predictor.

Why not just use mean, variance, and minimum pooling? By construction, these are blind to the manipulation in the controlled study, where all schedules share the same marginals. If humans rate those schedules differently, order-free pooling provably cannot explain it. If humans rate them the same, our hypothesis is false. The study is therefore designed so this alternative and our claim make opposite predictions.

Why not just rely on end-to-end models like DIVA-VQA or frame-consistency training? DIVA-VQA sees one-step differences, which register how much a frame changed but not whether the change belongs to a long episode. The frame consistency loss of Advancing Video Quality Assessment for AIGC acts on the score distribution, not its order. Whether these models already capture clustering implicitly is testable. We will regress their residuals on $\hat H$ and $\bar L$. If the residuals show no relationship, they already capture it, and our add-on will show no gain.

Why not just ask a vision-language model? VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness reports that such evaluators hallucinate, are sensitive to prompts, and need redundant question sets to be consistent. More to the point, they give no handle on temporal structure. We will include one as a secondary reference on the controlled set only, to see whether its scores change with schedule at fixed marginals.

Proposed Experiments

The first decisive check is the matched-marginal test on the controlled stimuli. Within source and distortion, we fit a mixed-effects model of MOS on schedule type and $\hat H$. The claim fails if the schedule effect is indistinguishable from zero. The non-monotone hypothesis fails if the quadratic term is not significant. In that case we fall back to whichever monotone direction the data show, either "flicker is worse" or "long episodes are worse."

The second check asks whether real videos show this structure at all. On KoNViD-1k, LIVE-VQC, YouTube-UGC, and the longer clips in T2VQA-DB, we estimate $\hat H$ for $v_t$ and for the signed series $q_t - q_{t-1}$. We check, as in the KLCI analysis, whether magnitude carries memory that sign does not. Following VBench's finding that models trading temporal consistency for motion collapse structurally (VBench: Comprehensive Benchmark Suite for Video Generative Models), we expect generated video to show higher $\hat H$ than natural video. We will report how many clips are too short for a reliable estimate instead of hiding them.

The third check measures incremental validity on human scores. We append the statistics to each baseline as described above and report SROCC and PLCC against MOS. The order-shuffled ablation must lose the gain; otherwise the improvement is not about time.

The fourth check targets the structural-break confound. On multi-shot clips, we compare $\hat H$ estimated with and without bootstrap change-point segmentation. We test whether unsegmented estimates inflate apparent persistence, and whether breaks the test flags inside single-shot generated clips match visible collapses identified by a human audit.

A positive result would give the field an interpretable, cheap temporal pooling rule and a validated stimulus set for temporal quality perception. A negative result on the matched-marginal test would be equally useful. It would mean the "unfairness of averages" is about how bad and how often, not when, and that order-aware machinery in video metrics is unnecessary.