Audio Tagging using Linear Noise Modelling Layer

Shubhr Singh, Arjun Pankajakshan, Emmanouil BenetosDCASE 2019 · OpenReview

Introduction

In machine learning, training labels are frequently treated as "ground truth." However, large-scale audio datasets often contain "noisy" labels derived from imprecise user tags or automated systems. This research explores an architectural approach to help deep neural networks (DNNs) mitigate the impact of incorrect training data by explicitly modelling the noise.

The Challenge of Label Noise

DNNs are highly capable of memorizing data, including errors. If a dataset contains mislabeled audio clips—such as a "Car Horn" labeled as "Dog Barking"—the network may learn these incorrect associations, leading to poor generalization. While manual verification is ideal, it is too costly for massive datasets.

This study utilizes the FSDnoisy18k dataset, where only 10% of the data is manually verified (clean), while the remaining 90% is "noisy" (unverified tags from Freesound). The goal is to develop a training strategy that remains robust despite the prevalence of incorrect labels.

Methodology: Linear Noise Modelling

The researchers employed a MobileNetV2 architecture and compared three training strategies. The primary contribution is the Linear Noise Modelling Layer.

The proposed architecture showing the training phase with the noise layer (top) and the inference phase without it (bottom).

The Noise Transition Matrix

The core idea is to add a simple linear layer with a softmax activation on top of the baseline network. This layer acts as a noise transition matrix, $p(z|y)$, where $z$ represents the observed noisy label and $y$ is the latent "true" label.

The workflow consists of:

  1. Pre-training: Training the baseline model on the small clean subset.
  2. Noise Learning: Initializing the noise layer with the baseline's predictions and retraining the entire stack on the noisy data to learn the "noise channel."
  3. Inference: During testing, the noise layer is removed. The model makes predictions using the underlying "clean" representation it learned before the noise layer corrupted the output.

Results and Analysis

The linear noise layer demonstrated superior performance compared to standard training and loss-based adjustments like soft bootstrapping.

Approach Test Accuracy
Baseline (no data aug) 64.9%
Baseline (with data aug) 66.7%
Baseline + Soft Bootstrapping 68.25%
Baseline + Linear Noise Layer 68.6%

Class-Specific Performance

The effectiveness of the noise modelling layer varied significantly across different audio categories.

Per-class accuracy comparison between the baseline and the noise modelling approach.

As shown in the figure above, the noise modelling layer provided substantial gains for classes with high noise levels, such as Clapping and Coin dropping. However, for other classes like Piano or Fire, the impact was neutral or slightly negative. This suggests that the effectiveness of the model depends on the specific structure and type of noise present in each category.

Key Insights

  • Noise as a Probabilistic Channel: By treating noise as a mathematical transformation, the network can effectively "back-calculate" the likely true label.
  • Architecture vs. Loss: Explicitly adding a physical layer to model noise proved more effective than simply modifying the loss function (e.g., bootstrapping).
  • Limitations: The current model assumes noise is "global" for a class. Future work aims to explore input-dependent noise models and multi-label audio tagging scenarios.

Conclusion

The introduction of a linear noise modelling layer provides a robust mechanism for training audio tagging models on imperfectly labeled data. By achieving a ~2% absolute improvement over the baseline, this approach demonstrates that explicitly accounting for label corruption is a viable strategy for large-scale, real-world audio analysis.

Summary updated

BibTeX
@inproceedings{singh2019audio,
  author = {Shubhr Singh and Arjun Pankajakshan and Emmanouil Benetos},
  title = {{Audio Tagging using Linear Noise Modelling Layer}},
  booktitle = {DCASE},
  year = {2019},
  url = {https://openreview.net/forum?id=RZwUTxsMpu},
}
Figure