Many audio tagging systems run in stages. A first model predicts something coarse, such as which instrument families are playing, and a second model reads those predictions alongside the audio to decide something finer or more abstract, such as which specific instrument it is or which genre the track belongs to. The second model is usually trained on the first model's predictions from the training set. Those predictions are cleaner than anything it will see at test time, so it learns to trust its parent too much. When the parent errs, the child repeats or amplifies the mistake. The classic remedy is stacking: train the child on held-out, cross-validated parent predictions so it sees realistic upstream errors (Stacked Hierarchical Labeling). We propose a cheaper alternative that comes with a working theory. During child training, we randomly corrupt the parent's outputs with noise whose covariance matches the parent's measured error covariance. Because this is noise injection, the existing theory of dropout applies. It says the corruption amounts to an explicit, adaptive weight penalty, and that penalty shrinks the child's reliance on each parent signal in proportion to how unreliable that signal is.
Open-licensed music audio is a natural place to test this. Copyright means much of music research works with frozen, shareable embeddings or pre-computed features, and in that setting only the prediction heads can be trained (Twenty-Five Years of MIR Research; Melon Playlist Dataset). Those heads are close to generalized linear models, which is the regime where dropout theory is exact. Open corpora also carry noisy, imbalanced labels and shift between collections. As a result, an upstream model's in-domain held-out error understates its real error on a different corpus. A corruption level that can be set and reasoned about is therefore more useful here than a fixed cross-validation split.
Core Claim
Consider a child head with logit $z = w^\top h + v^\top p$. Here $h$ is a frozen audio embedding, $p$ is the vector of soft predictions from the parent stage, and $w, v$ are the child's weights. We train on $\tilde p = p + \epsilon$. The noise $\epsilon$ has mean zero and covariance $\Sigma_k$ for parent class $k$, estimated from the parent's prediction residuals on a small calibration split. Dropout Training as Adaptive Regularization shows that unbiased noise of this kind, for logistic-type losses, is to second order the clean loss plus a quadratic penalty:
$$ \mathbb{E}_\epsilon[\mathcal{L}(\tilde p)] \approx \mathcal{L}(p) + \tfrac{1}{2}\sum_i \sigma'(z_i)\, v^\top \Sigma(p_i)\, v , $$
where $\sigma'(z_i)$ is the logistic curvature at example $i$.
The new step is the choice of $\Sigma$. In Wager et al.'s setting, and with ordinary isotropic dropout on the child's inputs, $\Sigma$ follows the variance of the features themselves. When $\Sigma$ is the parent's error covariance, the penalty on $v_k$ grows with parent $k$'s unreliability and is untouched by how often or how variably that parent fires. A rare instrument family that the parent detects reliably keeps its influence. A common family that the parent confuses is shrunk.
Our hypothesis is that this marginalized, confusion-matched penalty reproduces most of stacking's benefit. It needs one calibration split instead of K refits of the parent. It also exposes one interpretable knob: an inflation factor $\gamma$ that scales $\Sigma$ to the error expected at deployment rather than the error seen in-domain. If stacking and our method turn out equal in-domain, the remaining contributions are an explanation of why stacking works for linear heads and a cheaper way to do it. The cross-corpus setting is where we expect a clear advantage.
Research Program
The work has three parts.
The first is analytic. We derive the penalty above for multi-label logistic heads, including off-diagonal error correlations between sibling classes. Instrument families confuse each other in structured ways, and the full covariance captures this. We then compare the closed form against Monte Carlo corruption in order to locate where the second-order approximation fails. We expect it to fail when the parent's probabilities saturate near 0 or 1.
The second is a pair of cascades built entirely on Creative Commons audio. The in-domain cascade runs top-down on OpenMIC-2018. A family head predicts the coarse tone-base families defined by Chou et al., and an instrument head then reads the family posterior together with the embedding. The cross-corpus cascade runs bottom-up. An instrument detector trained on OpenMIC feeds a genre and mood head trained on MTG-Jamendo. MTG-Jamendo carries its own instrument tags, so we can measure the upstream detector's true error on the target corpus over the overlapping vocabulary and compare it with its in-domain held-out error. That comparison is how $\gamma$ is set and tested.
The third repeats both cascades across frozen embeddings evaluated in MARBLE, such as MERT and Musicnn, and with a lightweight non-linear (MLP) child head. The goal is to see whether the benefit survives when the probe is no longer close to linear.
Baselines
The naive reference is a cascade whose child is trained on the parent's own training-set predictions, with no corruption. This is the setting that produces over-trust. It anchors the "cascade amplification" metric defined below.
The strongest published method on OpenMIC hierarchy is ResAtt from An Attention-based Approach to Hierarchical Multi-label Music Instrument Classification. It trains fine and coarse levels jointly and uses learned attention to suppress spurious fine predictions before they reach family outputs. The paper reports coarse-level F1 of 85.5% against 84.4% for a flat baseline, and fine-level PR-AUC of 86.0% against 84.3% for a PaSST-S baseline. ResAtt fine-tunes its encoder, while our main setting uses frozen embeddings. We will therefore cite those figures as context and re-implement ResAtt's aggregation as a head on the same frozen embeddings, with the same tuning budget, so the comparison is fair. Beating it means matching its clean-test numbers while showing a smaller drop when the parent is degraded.
The most direct competitor on the actual question is deep stacking: K-fold (K=5) held-out parent predictions, following Munoz et al.. It gets the same embeddings and the same hyperparameter search budget for the child. Its parent refits are counted in a reported compute total. To beat stacking, our method must equal its in-domain child metrics and exceed them on the cross-corpus cascade.
The ablation that isolates the key ingredient is isotropic dropout on the child's parent-input channel, with the rate tuned on validation data. It has the same form of noise but the wrong covariance. A second ablation compares Monte Carlo corruption with the marginalized penalty, to test whether the closed form suffices. A third sets $\gamma = 1$, using only in-domain error, to test whether the inflation knob matters.
We found no reported result on Lacuna for MTG-Jamendo ROC-AUC or PR-AUC with these architectures. The only tagging numbers in the file are for MTAT, about 0.9129 ROC-AUC, and MSD, about 0.8898, from the MTG-Jamendo paper. Our Jamendo baselines will therefore be reported fresh.
Why not just…?
Why not just use stacking, since it already teaches the child to distrust its parent? Stacking is the real threat, and in-domain it may tie with our method. Its held-out predictions reflect only the parent's in-domain error. The cross-corpus cascade measures directly whether that underestimates error on Jamendo. If stacking still matches us there, we will report that the theory explains stacking's behavior for linear heads and offers it at roughly 1/K of the parent-training cost, and nothing more.
Why not just apply ordinary dropout to the child's inputs? The theory predicts it penalizes the wrong thing. Isotropic dropout scales shrinkage by feature variance (Dropout Regularization Versus l2-Penalization in the Linear Model), which is unrelated to how often a parent is wrong. The isotropic ablation tests this, and so does an inspection of whether the learned $|v_k|$ track parent reliability under each scheme.
Why not just train everything jointly end to end, as ResAtt does? Joint training needs a fine-tunable encoder and shared labels. Neither holds when only pre-computed features can be shared, or when the upstream model comes from a different corpus, as in the OpenMIC-to-Jamendo cascade. In-domain, the re-implemented ResAtt head is a full baseline, so if joint aggregation alone is enough, it will show.
Why not just calibrate afterwards with hierarchical conformal prediction? Conformal Prediction in Hierarchical Classification guarantees coverage but leaves the child model unchanged. A weak base model simply yields larger sets: about 44 at $r=3$ versus about 123 at $r=1$ on PlantCLEF 2015. The two approaches are complementary. We will run CRSVP on top of each child and report set sizes at 90% coverage, expecting smaller sets when the child's base model is stronger.
Proposed Experiments
The decisive check is a controlled degradation sweep. At test time we corrupt the parent's outputs at increasing error rates. We do this both synthetically and by swapping in parents trained on less data. We then plot child PR-AUC and macro-F1 against parent error and call the slope the cascade amplification. Our prediction is that the confusion-matched penalty yields the flattest slope at equal clean accuracy, and that isotropic dropout falls between it and the naive reference.
The second check is the cross-corpus cascade. We measure the gap between the OpenMIC detector's in-domain held-out error and its error on Jamendo's instrument tags, set $\gamma$ to close that gap using 200 labeled Jamendo tracks, and compare child genre and mood ROC-AUC and PR-AUC against stacking. If $\gamma > 1$ gives no gain over $\gamma = 1$, the shift argument fails.
The third check tests the theory itself. We compare closed-form against Monte Carlo training across embeddings and for MLP versus linear heads, which tells us how far the GLM derivation carries in practice. Every experiment uses CC-licensed audio and released embeddings, so the full pipeline can be redistributed.
References
- Stacked Hierarchical Labeling
- Twenty-Five Years of MIR Research: Achievements, Practices, Evaluations, and Future Challenges
- Melon Playlist Dataset: A Public Dataset for Audio-Based Playlist Generation and Music Tagging
- Dropout Training as Adaptive Regularization
- An Attention-Based Approach to Hierarchical Multi-Label Music Instrument Classification
- The MTG-Jamendo dataset for automatic music tagging
- MARBLE: Music Audio Representation Benchmark for Universal Evaluation
- Dropout Regularization Versus l2-Penalization in the Linear Model
- Conformal Prediction in Hierarchical Classification with Constrained Representation Complexity