Does Focused Listening Distort Noise Annoyance? Measuring It with Paired Ratings of the Same Sounds

Models that predict how annoying or pleasant an everyday sound is learn from people who were asked to sit down, listen closely to a short clip, and rate it. In daily life, people hear traffic, alarms, and neighbours while they are busy with something else, and nobody asks them about the noise. This proposal treats the gap between those two situations as a form of self-report bias that can be measured. When the rating task points attention at the sound and tells the rater the study is about noise, the rater is likely to report more annoyance than they would feel incidentally. We propose to measure this bias directly. The same urban clips will be presented under randomly assigned listening conditions, and we will then test whether a model can learn to translate "lab annoyance" into "incidental annoyance."

The central bet is that the distortion is not a constant offset but depends on the sound source. For example, intermittent alarms and horns may be over-rated relative to steady hums, which people tune out when attention is elsewhere. If that holds, existing annoyance predictors are miscalibrated in a way that simple rescaling cannot fix, and the correction needs to know which sound events are present. If it does not hold, a two-parameter recalibration is enough, which is worth knowing. The experiment is designed to tell these outcomes apart.

Core Claim

Current environmental-sound annoyance models are trained on labels collected under focused listening. These labels carry a systematic, source-dependent bias relative to how annoying the same sounds are when heard incidentally. A model with a shared audio encoder, separate prediction heads for each listening condition, and per-rater offsets can estimate that bias from a modest paired dataset. It can then predict incidental annoyance on held-out clips better than both direct transfer and global recalibration of a lab-trained model.

Where This Sits

Two papers motivate the proposal. Exploring Annoyance in a Soundscape Context by Joint Prediction of Sound Source and Annoyance trains a two-input CNN on Mel spectrograms and loudness trajectories using the DeLTA corpus of 2,890 15-second binaural urban clips. It names focused listening as the main threat to ecological validity but never measures it. Joint Prediction of Audio Event and Annoyance Rating in an Urban Soundscape by Hierarchical Graph Representation Learning links fine and coarse sound events to annoyance through a graph network on the same data. It shows that event identity carries most of the annoyance signal, but it treats the averaged lab rating as ground truth.

We reuse both papers' architectures as encoders and baselines. What we change is the target: we ask which of their labels' properties survive a change in listening condition.

Two other papers handle rater subjectivity and supply method we reuse. Automatic recognition of emotion evoked by general sound events builds consensus labels with the Evaluator Weighted Estimator, which weights each rater by agreement with the crowd. It also reports that disagreement clusters by sound category. That method corrects disagreement between raters within one rating condition. It cannot detect a bias every rater shares because they all sat in the same focused setup, and detecting that shared bias is our target.

A Sustainability-Oriented Assessment of Noise Impacts on University Dormitories shows that pleasantness splits by source at comparable measured levels and differs by gender. This supports including rater-level terms, but its cross-sectional survey design cannot separate what was heard from how it was reported.

From the self-report side, two papers constrain the design. Tag2Risk argues that demand characteristics, meaning participants guessing the study's purpose and answering accordingly, distort questionnaire answers. Yet its passive proxy is still validated against self-report scales. We avoid that circularity by adding a non-self-report anchor, described below. Assessing Real-Time Moderation for Developing Adaptive Mobile Health Interventions for Medical Interns shows that repeated in-the-moment questionnaires are burdensome and cause missing data, and that prompts themselves change what is reported. This warns us that our own "incidental" probes may make listening focused again, so we treat probe frequency as a factor to measure rather than a fixed design choice.

Research Program

Paired listening study. About 600 DeLTA clips will be rated, stratified across the seven coarse event categories, and they include the 578 official test clips so that results stay comparable with published models. Roughly 150 online participants wearing headphones (DeLTA is binaural) will each take part in two kinds of blocks, counterbalanced so that every clip receives ratings under every condition from different people.

In focused blocks, participants replicate the DeLTA protocol: listen to the clip, then rate annoyance from 1 to 10.

In incidental blocks, participants do a proofreading task framed as a reading study, while DeLTA clips play back to back as background. The debrief explains the framing, and the deception requires ethics approval. Within these blocks there are two kinds of report. An unannounced probe after a random subset of clips asks how annoying the background sound just was. A retrospective rating at the end of the block asks about representative clips that are replayed.

Throughout the incidental blocks we log proofreading errors and speed within each clip's time window. This gives a behavioral interference score that is not a self-report. It serves as a partial anchor for checking which form of report tracks actual disturbance.

The probe rate is varied between participants, either rare or frequent, to measure how much asking about noise turns incidental listening back into focused listening.

Model. Let $y_{r,c,k}$ be rater $r$'s rating of clip $c$ under condition $k$ (focused, incidental-probe, or incidental-retrospective). We fit

$$y_{r,c,k} = h_k(z_c) + b_{r} + \epsilon_{r,c,k},$$

where $z_c$ is the audio embedding from a pretrained audio network fine-tuned with event supervision as in the hierarchical graph paper, $h_k$ is a small prediction head for condition $k$, $b_r$ is a per-rater offset that captures scale use, and $\epsilon$ is residual noise.

The quantity of interest is the condition gap $\Delta(c) = h_{\text{inc}}(z_c) - h_{\text{foc}}(z_c)$. Our hypothesis is that $\Delta$ varies with event composition. The alternative is that $\Delta$ is affine in the focused prediction, meaning the same for every clip after a linear rescaling. Because only the head differs by condition, the focused head can be pretrained on the full DeLTA training set, and the incidental head learned from the much smaller paired data.

Analysis. First, a mixed-effects regression of rating on condition × coarse category, with random effects for rater and clip, tests for the interaction directly, without any neural model. Second, we test whether the behavioral interference score correlates more strongly with incidental reports than with focused ones, which would show that the focused labels are the more distorted of the two.

Baselines

The do-nothing reference applies a focused-trained model unchanged to incidental ratings, alongside a constant predictor that outputs the mean incidental rating. These two set the floor. Any proposed correction must beat direct transfer to show that the condition gap matters.

The strongest existing method is the hierarchical graph model, which reports an annoyance MSE of 1.049 on DeLTA versus 1.675 for a plain CNN (Hou et al. 2023). The two-input CNN reports RMSE 1.05, versus 1.27 for loudness alone (Hou et al. 2024). These numbers are on focused labels. We found no reported result on Lacuna for any model on incidental-condition annoyance.

We will retrain both models on the DeLTA training split with the same encoder initialisation and tuning budget, and confirm that we reproduce roughly their focused-label error on our re-rated test clips. We will then measure their error against our incidental ratings, which shows how far published numbers overstate real-world accuracy. A loudness-only model is included because a skeptic will argue that loudness explains incidental annoyance even if it does not explain focused annoyance.

Two label-cleaning baselines use the paired data with targets built either by the Evaluator Weighted Estimator or by z-scoring each rater, both applied within the incidental condition. These test whether better handling of rater noise alone closes the gap.

The key ablation replaces the event-aware incidental head with a two-parameter affine map on top of the frozen focused head. It is fit on exactly the same paired training clips with the same validation-based selection. The proposal counts as successful if the event-aware head lowers incidental RMSE on held-out clips and held-out raters by a margin whose bootstrap confidence interval excludes zero. A second ablation feeds the head ground-truth DeLTA event labels instead of predicted ones, which separates errors in event recognition from errors in the correction itself.

Why not just…?

Why not just run a field study with phone audio and momentary mood prompts? That is the natural next step for this gap, and our design does not replace it. In a field study, the sound heard, the person, and their activity are all mixed together: the people exposed to construction noise differ from those exposed to birdsong. Prompting also induces the reactivity documented in NeCamp et al.. Random assignment of identical clips to conditions isolates the effect of listening condition, which a field study cannot do. The fitted condition head is then a calibrated prior that a later field deployment can test.

Why not just weight or normalize raters, as the Evaluator Weighted Estimator does? That method fixes disagreement among raters within one setting (Schuller et al.). A bias shared by every focused rater survives it unchanged. The label-cleaning baselines test this directly. If they match the event-aware head on incidental error, our claim fails.

Why not just apply an affine recalibration fit on a few incidental ratings? This is the real threat. If the focused-listening bias is roughly a uniform inflation of annoyance, two parameters will capture it, and a multi-head model adds nothing. The affine ablation and the category × condition interaction test decide this. A null interaction together with affine parity would be reported as the result: lab annoyance labels would then be trustworthy up to a cheap recalibration, which is itself useful for noise-assessment practice.

Decisive Checks

The proposal stands or falls on three results. First, the condition × category interaction must be significant in the mixed model, with a predicted direction of focused annoyance being inflated more for intermittent events (alarms, horns) than for steady ones (hum, rain). Second, the event-aware head must beat affine recalibration on held-out incidental ratings. Third, incidental reports must correlate with proofreading interference more strongly than focused reports do.

The probe-rate manipulation adds a fourth reading. If frequent probes drive incidental ratings toward focused ones, then even in-the-moment questionnaires in field studies inherit part of this bias, which would bear on how field sound-affect data should be collected. Valence ratings may be added in the same sessions to check whether the pattern extends beyond annoyance, but that is secondary.