When people rate how good a phone call or a denoised recording sounds, they are judging several separate things at once. Is there leftover background hiss? Does the voice sound muffled or robotic? Are there dropouts? Today's automatic quality predictors such as NISQA and DNSMOS already estimate these separate perceptual dimensions. They then assume that a listener combines them into an overall judgment with fixed weights. That assumption has never been tested over the length of a real listening session, and drift is likely there. A listener who has just heard twenty clips full of steady hiss may stop noticing hiss and grow more sensitive to voice distortion, or the reverse. If that happens, the right trade-off for a noise suppressor depends on what the listener has recently heard.
This note proposes to measure and model that effect. The core idea is that each perceptual dimension carries its own exposure history. The weight a listener puts on noise, on coloration, or on discontinuity drifts at its own rate, driven by how much of that impairment the listener has recently heard. The work brings preference-drift modeling, which so far exists only for text and click data, into continuous auditory perception. It also adds something that work lacks: drift that differs by dimension rather than one discount applied to everything.
Core Claim
The claim is that over a session, a listener's overall quality rating is better predicted by a model in which each dimension's penalty weight evolves with recent exposure to that dimension than by either of two alternatives. The first is a static combination of dimension scores. The second is a single shared drift term, such as a drifting rating offset or uniform time-discounting. This is a falsifiable psychophysical claim with an engineering consequence: the right amount of noise suppression for a clip depends on what the listener heard before it.
Concretely, let $r_{u,t}$ be listener $u$'s overall rating of the $t$-th clip $x_t$ in a session. Let $I_d(x_t)$ be the impairment on dimension $d$ (noisiness, coloration, discontinuity), estimated by a frozen multidimensional predictor. The model is
$$r_{u,t} = b_u(t) - \sum_d w_{u,d}(t)\, I_d(x_t) + \epsilon_{u,t}, \qquad w_{u,d}(t) = w^0_{u,d} + \alpha_d\, e_{u,d}(t),$$
Here $b_u(t)$ is a slowly drifting rating offset, $w^0_{u,d}$ is the listener's resting weight on dimension $d$, $\alpha_d$ is a signed sensitivity, and $\epsilon_{u,t}$ is rating noise. A negative $\alpha_d$ means habituation and a positive one means fatigue or sensitization. The exposure state $e_{u,d}(t)$ is a leaky accumulator:
$$e_{u,d}(t) = \lambda_d\, e_{u,d}(t-1) + I_d(x_{t-1}).$$
The memory $\lambda_d \in [0,1)$ is fitted separately for each dimension. If every $\alpha_d$ is zero, the model reduces to the classic additive integration, which the file reports explains about 90% of overall-quality variance under mild degradations. If $\lambda_d$ and $\alpha_d$ are forced to be shared across dimensions, it reduces to a scalar drift model. Both reductions are the main comparisons.
Relation to Prior Work
Three papers motivate the gap. NISQA predicts overall MOS (mean opinion score, the average 1–5 rating) together with noisiness, coloration, discontinuity and loudness. Its attention pooling models recency, but only within a single few-second clip. DNSMOS P.835 predicts speech quality (SIG), background quality (BAK) and overall quality (OVRL) for noise suppressors. Both fix the mapping from dimensions to overall quality across clips. We keep their dimension predictors as frozen inputs and make the mapping depend on session history.
Two papers supply the temporal machinery. Right Now, Wrong Then (NS-DPO) down-weights older text preference pairs with a single exponential factor $\gamma^{T-t_i-1}$. That factor is shared by all criteria. Our per-dimension $\lambda_d$ generalizes it to differential drift, and NS-DPO's uniform discount becomes a baseline. FRec shows that fatigue in recommendation depends on how densely similar items were consumed, not on elapsed time alone. It models this with fatigue features gating a recurrent unit inside interest subspaces. Our exposure accumulator is the auditory analogue, with the interest subspaces replaced by perceptual dimensions.
Two papers constrain the design. SaD shows that splitting the training signal into BAK, SIG and overall discriminators matters for enhancement. On VoiceBank+DEMAND it lifted CMGAN from PESQ (a standard objective speech-quality score) 3.406 to 3.622. However, its weights adapt only to the current clip's signal-to-noise ratio and never to the listener. When Rubrics Backfire shows that hidden shifts in trade-off weights between criteria can distort a judge while benchmark agreement stays intact. This is a warning that aggregate correlation can hide weight drift, and our evaluation must measure weights directly.
The most important constraint comes from DNSMOS itself. Its correlations are 0.94–0.98 when averaged per system but only 0.71–0.83 per clip, with about five raters per clip. Clip-level noise of that size could easily be mistaken for drift. The experimental design below is built around this problem.
Research Program
The work has three parts.
First, we run a controlled listening study. The existing crowdsourced corpora (NISQA, DNS Challenge V3) were rated in sessions, but the file does not establish that per-rater presentation order is available. We therefore cannot assume that drift is recoverable from them. If order logs exist, we will analyze them as a secondary replication.
Stimuli come from the DNS Challenge V3 processed clips and the NISQA simulated corpora. Each clip is pre-scored by DNSMOS and NISQA so its dimension profile is known. Each listener rates P.835-style (SIG, BAK, overall) in sessions of about 150 clips. Sessions are built from blocks dominated by one impairment type: a noise-heavy block, a coloration/distortion-heavy block, a discontinuity-heavy block, and balanced control blocks. Block order is counterbalanced across listeners.
The key device is a fixed set of anchor clips with known profiles. These are re-inserted at many session positions and after different block types. Because the same audio is rated repeatedly, any systematic change in its rating is drift, not clip-to-clip label noise. A noise-dominant anchor rated after a noise block versus after a coloration block gives a direct, model-free test of dimension-specific drift. A matched set of short, spaced-out sessions serves as a no-accumulation control, in the spirit of FRec's finding that spacing removes fatigue.
Second, we fit and compare session models. These are the exposure model above, its scalar reductions, and the flexible sequence model described under Baselines. All are fitted per listener with hierarchical pooling across listeners. They are evaluated on held-out later portions of sessions and on held-out listeners. We also test the non-linear "one catastrophic dimension dominates" regime by adding an interaction term. That term asks whether accumulated exposure shifts a listener from additive integration into single-dimension dominance.
Third, we test the consequence for enhancement. We take one noise suppressor with a single strength control that trades residual noise (BAK) against voice distortion (SIG). It can be any of the DNS-style systems whose outputs are in the stimulus pool, or CMGAN with a mixing parameter. In new sessions, the strength for each clip is chosen either at a fixed value tuned for population-average DNSMOS OVRL, or by maximizing the exposure model's predicted rating given that listener's history so far. Human overall ratings decide which wins.
Baselines
The naive reference is the static DNSMOS P.835 OVRL prediction for each clip, mapped to each listener's scale with a per-listener linear offset and slope. This is what practitioners do today. The strongest published number for it is the clip-level PCC (Pearson correlation) of 0.71–0.83 reported in DNSMOS P.835. NISQA reports dimension RMSE of 0.16–0.56 across datasets. No reported result was found on Lacuna for any predictor on session-ordered, per-listener rating sequences. Our study will therefore report every baseline on the same new data.
The static multidimensional baseline fits each listener's own fixed weights $w^0_{u,d}$ on the frozen dimension scores. This is the additive integration model and the strongest competitor if no dimension-specific drift exists. It already captures individual differences, which is the role P-ShareLoRA plays for text annotators.
The scalar drift baselines are of two kinds. One is a random-walk rating offset $b_u(t)$ with static weights, which tests whether drift is just response-scale creep. The other refits the static weights under NS-DPO-style exponential discounting with one $\gamma$ shared by all dimensions, as in Right Now, Wrong Then.
The flexible sequence baseline is a small recurrent network. It reads the history of dimension scores and the listener's past ratings and predicts the next rating. It is in the spirit of FRec, but without our per-dimension structure.
The key ablation keeps our model but ties $\lambda_d$ and $\alpha_d$ across dimensions. This isolates the "per-dimension" ingredient.
All models receive the same frozen DNSMOS/NISQA dimension inputs, the same train/test splits, and the same hyperparameter search budget, selected by validation likelihood. To beat the baselines, the exposure model must improve held-out log-likelihood and next-rating RMSE over both the static multidimensional baseline and the tied ablation. The improvement must appear on held-out listeners and on anchor clips. In the enhancement test, history-adaptive strength must earn higher human overall ratings than the fixed setting under the same listeners and stimuli, with a pre-registered paired test.
Why not just…?
Why not just track a single time-discounted overall score? A scalar drift model can absorb a listener becoming harsher overall. It cannot say which trade-off changed, and the trade-off is what a noise suppressor must act on. DNSMOS P.835 notes that equal overall scores can come from opposite behaviors: heavy suppression with distortion, or light suppression with residual hiss. The anchor-crossover test settles this directly. If a noise-dominant anchor and a distortion-dominant anchor shift in opposite directions after the same block, scalar drift is ruled out. If they shift together, scalar drift is enough and we will report that.
Why not just fit a static per-listener weighting? This is a real threat. It is plausible that listeners differ a lot from one another but are stable within a session, in which case this baseline wins. The spaced-session control and the anchor clips measure within-session change directly. A null result there would be a useful finding, because it would justify the fixed-mapping assumption that NISQA and DNSMOS rely on but have never tested.
Why not just give a recurrent network the full history? With enough data, a flexible sequence model can match any drift pattern. On per-listener sessions of a few hundred ratings, though, it is likely to overfit, and it yields no interpretable dimension rates that could steer a suppressor. We will run it anyway. If it beats the exposure model on held-out listeners, we adopt its predictions and use the structured model only for interpretation. If the two tie, the structured model is preferred because its few parameters can be checked against the anchor measurements.
Why not just ask a frontier audio-capable language-model judge to adjust for context? The file gives reasons for doubt. In AcCoRD, prompted frontier agents reached under 30% perfect outcomes when preferences drift. When Rubrics Backfire shows that prompt-specified trade-offs drift in unmonitored ways. TRACE indicates that language-model judges need extra grounding to read acoustics at all. We treat this as out of scope rather than as a baseline, since it would not use the listener's own history.
Decisive Checks
The first and decisive check is the anchor crossover. We compare ratings of noise-dominant and distortion-dominant anchors after noise-heavy versus distortion-heavy blocks. A significant block-by-anchor interaction, larger than the test-retest spread of anchors within balanced blocks, supports dimension-specific drift. No interaction falsifies the core claim.
The second check is the spacing control. The interaction should shrink in spaced sessions if the cause is accumulated exposure rather than block identity.
The third check is model comparison on held-out listeners against the tied ablation.
The fourth check is the human-rated enhancement test, which is the practical payoff.
As a secondary, exploratory question, we will ask whether listener-history effects contribute to the corpus-dependent rank changes reported in ConferencingSpeech 2022. Corpora dominated by different impairments would induce different exposure states in their raters. We flag this as a hypothesis, not a result.
References
- NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets
- Dnsmos P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors
- Right Now, Wrong Then: Non-Stationary Direct Preference Optimization under Preference Drift
- Modeling User Fatigue for Sequential Recommendation
- SaD: A Scenario-Aware Discriminator for Speech Enhancement
- When Rubrics Backfire: Systematic Preference Drift in LLM Judges
- A Shared Low-Rank Adaptation Approach to Personalized RLHF
- AcCoRD: Evaluating User-Agent Collaboration Under Realistic User Preference Dynamics
- Hearing Between the Lines: Unlocking the Reasoning Power of LLMs for Speech Evaluation
- ConferencingSpeech 2022 Challenge: Non-intrusive Objective Speech Quality Assessment (NISQA) Challenge for Online Conferencing Applications