Detectors that recognise overlapping sounds, such as many birds calling at once or a door slamming over speech, are trained offline and then deployed on audio streams whose noise, species mix and call density drift over hours and days. The standard repair is to pick a decision threshold per class on held-out data and smooth the output over time. Both steps are fixed once and never revisited. The BirdSet benchmark shows the cost. A ConvNeXt trained on 6,800 hours of clean single-bird recordings reaches only 0.36 class-mean average precision and 51% top-1 accuracy on real multi-species soundscapes (BirdSet). We propose to keep the large detector frozen and attach a tiny corrector to it. For each class, the corrector rescales that class's score and adds a learned contribution from the scores of the classes it most often overlaps with. The corrector is updated online from the sparse labels a monitoring annotator verifies over time.
The theoretical tool is online convex optimization: a learner commits to a decision each round, then sees a convex loss. The relevant results show that when losses are "curved", a learner's cumulative loss relative to the best decision in hindsight (its regret) grows like $\log T$ rather than $\sqrt{T}$. Curved here means bowl-shaped, as the logistic log-loss is on a bounded domain. Our observation is that in overlapping audio this curvature is not spread evenly over time. The log-loss curvature of a class at a frame is proportional to $p(1-p)$, where $p$ is the predicted probability that the class is active. That quantity is near zero in silence and in clean single-source calls. It is large exactly in the ambiguous, masked, overlapped frames where detectors fail. The corrector therefore learns fastest where the overlap problem lives. This yields a sharp, testable prediction: curvature-exploiting online learners should beat flat gradient methods by a margin that grows with polyphony.
Core Claim
A frozen detector plus a low-dimensional per-class corrector, trained with a discounted curvature-exploiting online learner, can track soundscape drift with provable dynamic-regret guarantees and no per-site tuning. Its gains should concentrate in high-overlap frames. Write $s_{t,c}$ for the frozen detector's logit for class $c$ at frame $t$, and $N(c)$ for the $K$ classes that most often co-occur with $c$ in training data. The corrector predicts
$$p_{t,c} = \sigma\Big(a_{t,c}\, s_{t,c} + b_{t,c} + \sum_{k \in N(c)} \gamma_{t,ck}\, s_{t,k}\Big),$$
with parameters $w_{t,c} = (a_{t,c}, b_{t,c}, \gamma_{t,c\cdot}) \in \mathbb{R}^{K+2}$ constrained to a ball of radius $B$. Here $\sigma$ is the logistic sigmoid. The loss on each labelled frame is binary log-loss. Performance is measured by dynamic regret against any slowly moving sequence of correctors $u_1,\dots,u_T$ with path length $P_T=\sum_t\|u_t-u_{t-1}\|$.
Dynamic Regret via Discounted-to-Dynamic Reduction proves that discounted follow-the-regularized-leader on logistic losses achieves $O(dB\log(BT)+\sqrt{dBTP_T})$ dynamic regret without the exponential dependence on $B$ that usually plagues exp-concave methods. That paper supplies the method. In our setting $d=K+2$ per class, so the guarantee is meaningful even at hundreds of classes.
What is new is not the bound. It is the use of this machinery as an overlap-aware adaptation layer for audio, together with an empirical and analytical account of where the curvature comes from in bursty acoustic streams. A useful fact sharpens the evaluation. Per-class average precision is invariant to any fixed monotone transform of that class's score. A static recalibration therefore cannot move class-mean average precision at all, and any gain we report must come from adaptation over time or from cross-class coupling.
Research Program
We first build the corrector and three online learners for it. The first is discounted follow-the-regularized-leader on log-loss, as in the reduction of Xie et al. The second removes the need to choose a discount factor by aggregating over a continuum of discounts, following Discounted Online Convex Optimization: Uniform Regret Across a Continuous Interval. That paper supplies the tuning-free mechanism we rely on to claim one setting works across all sites. The third is the Online Newton Step of Logarithmic regret algorithms for online convex optimization without discounting, which captures curvature but assumes a fixed comparator.
Updates happen only on frames whose labels are revealed. We simulate an annotator who verifies a fraction $\rho \in \{1\%, 5\%, 20\%\}$ of windows, chosen at random. Active selection in the style of Towards Deep Active Learning in Avian Bioacoustics is an optional later extension, not part of the core claim.
Second, we analyse curvature empirically. For every class and site we log the accumulated curvature $\sum_t p_{t,c}(1-p_{t,c})$ and relate per-class regret to it. The analytical question is whether the $\log T$ term in the bound can be restated as the log of accumulated curvature. If so, regret would scale with the number of ambiguous frames rather than stream length.
This connects to a tension in the literature. Following the Leader and Fast Rates in Online Linear Prediction shows fast rates from curved decision sets collapse when the average loss vector nears zero, its "non-singularity" condition. That paper constrains our design. Long silent stretches in passive acoustic monitoring are exactly such near-zero regimes. We therefore take curvature from the loss rather than from the decision set, and we test directly whether learning stalls during silence and resumes during choruses.
Third, we probe the overlap mechanism. Polyphonic Audio Event Detection: Multi-Label or Multi-Class Multi-Task? captures inter-event dependence by treating combinations of events as classes. To keep the number of combinations manageable, it hand-partitions the vocabulary, which its authors leave as an open problem. Our coupling terms $\gamma_{ck}$ are a continuous, linear-cost substitute for that combinatorial structure. Unlike their fixed offline partition, they adapt to whichever co-occurrences the current site exhibits. Unlike the independent-label assumption that 3MAS explicitly flags as a simplification, they let a dominant sound's score shift the evidence threshold for the sounds it masks. That masking effect motivates the coupling.
Baselines
The naive reference is the frozen detector with per-class thresholds tuned once on validation data and a median filter on the binarised output. We use ConvNeXt trained in BirdSet's large-training regime, the best model there with mean AUROC 0.85 and cmAP 0.36 across the eight soundscape test sets (BirdSet). On TUT-SED-Synthetic-2016 the frozen detector is a convolutional-recurrent network. Its strongest structural competitor is the multi-task combination model of Phan et al., which reports up to 3.7% absolute F1 over a multi-label baseline with 8 tasks. No reported absolute F1 for that model appears in the material we have, so we will retrain it and report our own numbers.
The strongest competitor on our actual question, adapting to drift from sparse labels, is an exponentially weighted sliding-window refit of Platt-style per-class recalibration, with neighbour scores included as inputs. It is essentially discounted follow-the-leader and needs only a window length. We give it every advantage. Its window is tuned per site in hindsight in one variant and tuned on one site and transferred in another, and it sees the same labelled frames and the same features. We found no reported result on Lacuna for online recalibration of BirdSet detectors.
The flat-gradient comparison is projected online gradient descent and Adam on the identical corrector and loss, with learning rates tuned on a held-out site. Adam is included because Xie et al. show it is itself a discounted follow-the-regularized-leader variant, so it is the closest "practical" learner.
Two ablations isolate the key ingredients. Setting $\gamma=0$ removes cross-class coupling and leaves only per-class drift tracking. Undiscounted Online Newton Step removes the ability to track a moving comparator.
Beating the field means three things on the soundscape test sets. Our method must exceed the transferred sliding window on cmAP and cumulative log-loss on a majority of sites. It must match the per-site hindsight-tuned window within its run-to-run variation. Its advantage over online gradient descent must grow with ground-truth polyphony.
Why not just…?
Why not just tune thresholds offline and smooth? Static per-class transforms leave per-class average precision unchanged, so they cannot address the 0.36 cmAP ceiling at all. Instrument Activity Detection in Polyphonic Music shows that smoothing over 5–10 second windows buys stability by giving up temporal resolution. That paper also names adaptive thresholding as an open need. The naive baseline is included to make this gap visible rather than assumed.
Why not just refit Platt scaling on a sliding window? This is the real threat, because it is nearly the same estimator as discounted follow-the-leader. Our only claimed advantages are two. The discount is chosen automatically by aggregation across discount factors, and the method carries a regret guarantee under drift. If a window tuned on one site transfers as well as our tuning-free learner across all eight BirdSet regions, we will report that the theory adds a guarantee but no practical gain. The window-transfer experiment settles this directly.
Why not just fine-tune the detector online, or test-time adapt with Adam? Updating millions of weights from a few percent of labelled frames on a bursty stream risks forgetting and is expensive on edge recorders. Adam on our small corrector is already a baseline, which separates the choice of learner from the choice of how much to adapt. We will run backbone fine-tuning at the 20% label rate as an upper reference, at matched label budget.
Why not just separate sources first? Improving Bird Classification with Unsupervised Sound Separation shows separation front-ends help. Separation is complementary to our method, not a competitor. We will run the corrector on top of separated-input scores on one site to check whether the gains stack.
Proposed Experiments
The primary testbed is BirdSet's eight strongly labelled soundscape sets, streamed in recording order, with the frozen large-training ConvNeXt. We report cmAP, AUROC, top-1, cumulative log-loss, and dynamic regret against the best piecewise-constant corrector fitted in hindsight. All results are given per site and per label rate, with five random label-reveal seeds.
The controlled testbed is TUT-SED-Synthetic-2016, where up to six events overlap. Here we stratify segment F1, error rate and output switch counts by the number of simultaneously active events. We also inject synthetic drift by remixing the 20 evaluation mixtures with rising background gain.
The decisive checks are three. First, the gap between curvature-exploiting learners and online gradient descent should increase monotonically with polyphony and with accumulated curvature. If it is flat, the central mechanism is wrong. Second, the coupled corrector should beat the $\gamma=0$ ablation mainly on frames where a high-scoring co-occurring class is present, which tests the masking hypothesis. Third, per-class regret plotted against log accumulated curvature, rather than against $\log T$, should collapse across sites onto a single trend. That collapse would support the curvature-count refinement of the bound.
A smaller transfer study applies the same corrector to overlapped-speech detection on AMI and DIHARD scores from a 3MAS-style model. It asks whether the polyphony-dependent gain appears outside bioacoustics.
References
- BirdSet: A Large-Scale Dataset for Audio Classification in Avian Bioacoustics
- Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer
- Discounted Online Convex Optimization: Uniform Regret Across a Continuous Interval
- Logarithmic regret algorithms for online convex optimization
- Towards Deep Active Learning in Avian Bioacoustics
- Following the Leader and Fast Rates in Online Linear Prediction: Curved Constraint Sets and Other Regularities
- Polyphonic Audio Event Detection: Multi-Label or Multi-Class Multi-Task Classification Problem?
- 3MAS: a multitask, multilabel, multidataset semi-supervised audio segmentation model
- Instrument Activity Detection in Polyphonic Music using Deep Neural Networks
- Improving Bird Classification with Unsupervised Sound Separation