Subtracting the Expected Sound: Context-Dependent Popularity for Urban Sound Tagging and Noticing

People stop hearing the sounds they hear all the time. A bus engine outside an office window fades from awareness within minutes, while a single unexpected shout gets through. Acoustics research formalizes this as the notice-event model: a sound disturbs people when it is noticed, and it is noticed when it stands out against what the listener has come to expect. Recommender-systems research has spent years on a problem with the same shape. Models trained on interaction logs mistake "seen often" for "relevant", and the main fixes estimate an item's popularity and subtract its influence from the score. This note proposes treating the expected sound at a given place and time as a popularity term. We would learn it explicitly, keep it when the goal is accurate tagging, and subtract it when the goal is to find what a resident would actually notice.

Concretely, we build an urban sound tagger whose score for each class is the sum of two parts. One part comes from the audio. The other is a learned, time- and place-varying prior: how often this class occurs at this sensor at this hour of this weekday. The idea comes from counterfactual debiasing in recommendation, with one change that matters in both fields. The popularity term is not one corpus-wide number per item. It depends on context and shifts over time. We test two claims on the SONYC sensor-network data. The first is that this factorized design uses spatiotemporal context where simple concatenation did not. The second is that dialing the prior down at inference produces an "unexpectedness" score that finds atypical events and predicts human noticing better than raw event probabilities or acoustic level contrast alone.

Core Claim

SONYC-UST-V2 attached block-level location and hour/day/week to every clip. Its authors found that appending these features to OpenL3 audio embeddings in a multilayer perceptron gave "minimal performance gains" over the context-free model, with a coarse macro-AUPRC (area under the precision-recall curve, averaged over classes) of 0.49 against 0.69 for human annotators (SONYC-UST-V2). We think a large part of that failure is structural. Much of what context tells you about a clip is a class prior: what is usually heard here, now. An MLP fed concatenated one-hot vectors has to rediscover that prior from about 18,000 clips spread over thousands of sparse sensor-by-hour cells. And because the SONYC-UST-V2 test audio was recorded after the training audio, any memorized cell-level quirks go stale.

Recommender debiasing faced the same entanglement between a "how common is this item" signal and a "does this match" signal. Model-Agnostic Counterfactual Reasoning (MACR) separates the score into an item-only popularity branch and a user–item matching branch, trains both, and at inference subtracts a scaled item-only effect. MACR, like almost all debiasing methods in the file, treats popularity as a static property computed over the whole corpus. The research file lists this explicitly as an open limitation of the field.

We borrow MACR's decomposition and replace static popularity with a context-indexed one. For class $c$, audio clip $x$, and context $k$ (sensor, hour of day, day of week), the logit is

$$z_c^{\lambda}(x,k) = f_c(x) + (1-\lambda)\, g_c(k),$$

Here $f_c$ is the audio-evidence branch and $g_c$ is the context-prior branch. The scalar $\lambda$ is set at inference. With $\lambda=0$ the model is a context-aware tagger. With $\lambda=1$ it reports audio evidence with the prior removed. With $\lambda>1$ it reports surprise: evidence for a class beyond what is expected at that place and time. To let the prior generalize across sparse cells, we make it low-rank:

$$g_c(k) = \mathbf{w}_c^{\top}\left(\mathbf{e}_{\text{sensor}} + \mathbf{e}_{\text{hour}} + \mathbf{e}_{\text{day}}\right),$$

The class vectors $\mathbf{w}_c$ and the context embeddings $\mathbf{e}$ are shared across all cells. This sharing is the part that concatenation lacks: it lets a rarely recorded sensor at 3 a.m. inherit structure from similar sensors and nearby hours.

The new element is a single learned object, the time-varying expected soundscape. It is used in two opposite ways and evaluated against both tagging accuracy and human noticing. The prior-subtracted mode is our operational version of the notice-event idea from Botteldooren et al. (2005). In that model annoyance is mediated by whether an event breaks through attention. Here, breaking through is defined relative to a listener's long-run exposure, not relative to the clip alone.

Relation to Prior Work

Two strands motivate the gap. On the acoustics side, Botteldooren et al. (2004) and Bockstael et al. (2012) show that perception tracks temporal structure and that recurrence governs habituation. Rhythmic sounds that keep resetting attention annoy more. Terroir et al. (2013) show that an event's dominance over the ambient background decides whether it reaches awareness. All of these define salience within a listening episode. None use the long-run statistics of a place, which is what habituation over weeks would depend on. On the machine-learning side, the SONYC datasets (SONYC-UST, SONYC-UST-V2) provide that long-run context but use it only through concatenation, and they report weak performance on infrequent classes such as music and non-machinery impact.

Other soundscape models touch context or skew but not their interaction. Hierarchical graph representation learning (HGRL) jointly predicts events and annoyance on 15-second DeLTA clips with a static event-to-annoyance graph. It has no notion of what is typical at the recording site. PSM folds hour and month into probabilistic audio–image–text embeddings and names data bias as future work. It conditions on context but never separates "common here" from "present in this clip", and separating those two is the point of our design.

From recommendation we take the method and some constraints. MACR supplies the two-branch training and inference-time subtraction. CLRec shows that contrastive losses act as implicit inverse propensity weighting, and MACR notes that explicit weighting is high-variance and hard to tune. Both warn us against relying on reweighting alone. Adap-τ argues that frequent items accumulate large embedding norms that dominate inner-product scores. That gives us a cheap ablation, normalizing the class weights of $f$, so that "the prior did the work" can be separated from "the norms were fixed". CSBR warns that uniformly boosting tail items hurts accuracy compared with context-calibrated corrections. This is the recommender version of our argument that the right correction depends on context.

Research Program

The first component is the tagger. We train it on SONYC-UST-V2 with frozen OpenL3 features (matching the published baseline) and with a fine-tuned audio encoder. Following MACR, we use three binary cross-entropy terms: one on the full logit $z^0$, one on the prior branch $g$ alone so that it is pushed toward the class base rates, and one on the audio branch $f$ alone so that it remains a usable detector without context. We also fit a temporal-drift variant in which day-of-week and a coarse calendar index enter $g$. This tests whether an explicit, smooth prior extrapolates better across SONYC-UST-V2's chronologically later test split.

The second component is an "atypical event" evaluation built from SONYC-UST-V2 metadata. A test positive for class $c$ counts as atypical when its sensor-and-hour-block cell lies in the bottom quantile of training frequency for $c$. We use hierarchical shrinkage so that empty cells fall back to the sensor, then to the hour, then to the global rate. These cells are where a habituated listener would be surprised and where a prior-heavy tagger would be most tempted to miss.

The third component is a modest perceptual study, needed because no dataset in the file pairs long-run site history with human judgments. DeLTA has annoyance ratings but no site histories, and SONYC has histories but no ratings. We would play listeners a compressed, time-ordered sequence of SONYC clips from one sensor, standing in for a day at that location, and then ask them to rate the final clip for noticeability and annoyance. The same final clip appears after histories from different sensors or hours. This varies its expectedness while holding the audio fixed, which isolates the habituation effect the notice-event model predicts.

The fourth component is a short transfer check in the other direction. On Last.fm, which CSBR uses with an 80/20 head/tail split, we replace MACR's static popularity branch with an hour-of-day-indexed one. We ask whether context-dependent popularity also improves the trade-off between tail exposure and accuracy there. This is secondary. It matters only because it tests whether the static-popularity limitation is real in recommendation as well.

Baselines

The naive reference point is the context-free audio model from SONYC-UST-V2, an MLP on OpenL3 embeddings. Alongside it is the published concatenation baseline, the same MLP with one-hot spatiotemporal features appended, which reports a coarse macro-AUPRC of 0.49 (SONYC-UST-V2). The file gives no separate number for the context-free variant, only that adding context gave minimal gains. We will reproduce both on the official splits with the same features, hidden sizes, and a matched hyperparameter search of equal trial count, and report our reproduced numbers next to the published 0.49. For the original SONYC-UST split, the published logistic-regression-on-VGGish baseline reaches 0.76 coarse and 0.62 fine AUPRC (SONYC-UST). We include it as a sanity anchor, not as a target.

The strongest competitor, in our view, is not a published model. It is a count-based prior correction: estimate $P(c \mid \text{sensor}, \text{hour}, \text{day})$ from training labels with hierarchical shrinkage, then add its log-odds to the context-free logits. That is our model with a fixed $g$ in place of a learned one. It is the obvious thing a careful practitioner would try, and it directly tests whether learning the prior is necessary. We tune its shrinkage strengths on the validation split with the same trial budget as our model. For the surprise mode we compare against subtracting this count prior. The proposal beats it only if the learned low-rank prior gives higher overall macro-AUPRC on the temporally displaced test set and higher AUPRC on the atypical subset, in both cases with confidence intervals over five seeds that exclude zero difference.

For rare classes we add class-frequency inverse weighting and focal loss on the context-free model, tuned equally. We also add the straight MACR transfer, in which $g$ depends on the class only and not on context. The ablations are a learned $g$ without low-rank sharing (a free parameter per cell), our model without the auxiliary branch losses, and our model with Adap-τ-style normalized class weights in $f$. For the perceptual study, the reference predictors are the context-free event probabilities, the clip's level contrast against its own background in the spirit of Terroir et al. (2013), and $L_{Aeq}$. For the Last.fm check, MACR and CSBR are the baselines. CSBR reports an 18.83% relative increase in Tail@20 on Last.fm. The file gives no absolute Tail@20 or Recall values, so we will rerun CSBR ourselves rather than compare against a quoted number.

Why not just…?

Why not just add a smoothed count-based log-prior to the logits? This is the real threat, and it is built into the design as the headline baseline. The case for learning the prior is sparsity and drift. SONYC-UST-V2 averages only a handful of clips per sensor-hour-week cell, and its test data comes from later in time. A shared low-rank prior can pool across similar sensors and hours, and it trains jointly with the audio branch so that the two branches do not double-count evidence. If the tuned count prior matches us on both overall and atypical AUPRC, we will report that. The contribution then narrows to the evaluation finding: an explicit prior, however obtained, rescues context where concatenation failed. Subtracting it then yields a usable noticing signal.

Why not just reweight rare classes or use focal loss? Reweighting changes how much each class's errors count, but it uses one global weight per class. It cannot express that music is rare on a weekday morning near a construction site yet common on a Saturday night elsewhere. That place-and-time dependence is what both the atypical-event metric and the notice-event model care about. MACR and CLRec also report that explicit inverse-propensity weighting is high-variance and hard to tune. The tuned reweighting baselines will show whether gains on the atypical subset come from context or merely from emphasizing rare classes. If reweighting matches us on the atypical subset, the context claim fails.

Why not just measure acoustic contrast or level, which is what perception studies already use? Within-clip contrast against background, the kind of dominance Terroir et al. (2013) study, may well explain most noticing. Energy averages are known to miss temporal structure (Botteldooren et al., 2004), but they are still a fair baseline. The perceptual study settles this with a nested comparison. We first fit noticeability on level contrast and $L_{Aeq}$, then add our surprise score, using mixed-effects models with random effects for listener and clip. Our hypothesis gains support only if the surprise score adds explained variance. The key condition is when the same audio is preceded by different histories. In that condition acoustic contrast is constant by construction, so any effect must come from expectation.

Proposed Experiments and Decisive Checks

On SONYC-UST-V2, the first decisive check is whether $z^0$ beats both the reproduced concatenation MLP and the tuned count prior in coarse and fine macro-AUPRC. We break results down by how far each test clip falls from the training period, to see whether the advantage grows with temporal distance as the drift argument predicts.

The second check is the $\lambda$ sweep. Tagging AUPRC on typical cells should fall as $\lambda$ rises, while AUPRC on atypical cells should rise until somewhere past $\lambda=1$. The crossing point is reported per class. A flat curve would mean $g$ has absorbed nothing that distinguishes typical from atypical occurrences, and that would falsify the design.

The third check is the listening study described above. Its decisive contrast is the within-clip effect of history on noticeability ratings, tested against the prediction of $g$ for that history.

As an exploratory check on DeLTA, we add a corpus-level class-prior branch to an HGRL-style annoyance head. We ask whether subtracting it changes predicted annoyance mainly for vehicle-heavy clips. HGRL reports an annoyance mean squared error of 1.049 against 1.675 for a plain CNN (HGRL). We treat 1.049 as the bar to stay at or below, not as something we expect to beat, since DeLTA has no site histories with which to test the temporal claim.

The Last.fm transfer is the last check. It asks only whether an hour-indexed popularity branch improves MACR's tail-versus-recall curve. A positive result would suggest that context-dependent popularity is a common fix for both communities, not a trick specific to sound.