Abstract
Music auto-tagging is crucial for enhancing music discovery and recommendation. Existing models in Music Information Retrieval (MIR) struggle with real-world noise such as environmental and speech sounds in multimedia content. This study proposes a method inspired by speech-related tasks to enhance music auto-tagging performance in noisy settings. The approach integrates Domain Adversarial Training (DAT) into the music domain, enabling robust music representations that withstand noise. Unlike previous research, this approach involves an additional pretraining phase for the domain classifier, to avoid performance degradation in the subsequent phase. Adding various synthesized noisy music data improves the model's generalization across different noise levels. The proposed architecture demonstrates enhanced performance in music auto-tagging by effectively utilizing unlabeled noisy music data. Additional experiments with supplementary unlabeled data further improves the model's performance, underscoring its robust generalization capabilities and broad applicability.
Introduction
Music auto-tagging—the automated assignment of descriptive labels such as genre, mood, or instrumentation—is a foundational technology for modern streaming services. While current models perform exceptionally well on clean studio recordings, their accuracy often collapses when applied to "in-the-wild" audio. Real-world multimedia, such as social media clips or amateur videos, frequently contains music layered with environmental noise, speech, or applause.
The core challenge lies in the "domain gap": standard feature extractors produce significantly different embeddings for a clean track compared to its noisy counterpart. This research addresses this gap by employing Domain Adversarial Training (DAT) to force the model to learn noise-invariant representations, ensuring that the underlying musical features remain identifiable regardless of the acoustic environment.
Methodology
Architectural Components
The proposed framework consists of three primary modules:
- Feature Extractor (FE): Utilizing a SampleCNN backbone and CLMR (Contrastive Learning of Musical Representations), this module transforms raw audio into high-dimensional embeddings.
- Domain Classifier (DC): A network designed to distinguish whether an embedding originated from a clean or a noisy audio source.
- Label Predictor (LP): The final classification layer that predicts specific music tags from the embeddings.
The Three-Step Training Strategy
To prevent the performance degradation often associated with simultaneous adversarial training, the authors developed a staged approach:
- Step 1: FE Pretraining: The Feature Extractor is trained using contrastive loss to identify general musical features across both clean and synthesized noisy audio.
- Step 2: DC Pretraining: The Feature Extractor is frozen, and the Domain Classifier is trained to identify the presence of noise within the embeddings. This establishes a strong baseline for the classifier before the adversarial phase.
- Step 3: Adversarial Finetuning: This stage employs a Gradient Reversal Layer (GRL). The FE is optimized to predict tags accurately via the LP while simultaneously attempting to "fool" the DC. This process strips away domain-specific noise information, resulting in embeddings that appear identical to the DC whether the input is clean or noisy.
Experimental Setup
Datasets
The study leveraged several key datasets to ensure robustness:
- MTG-Jamendo: Used for initial pretraining of the Feature Extractor.
- MagnaTagATune (MTAT): The primary benchmark for the downstream auto-tagging task.
- Audioset & Musan: Sources of real-world non-music noise (speech, environmental sounds) used to synthesize noisy training and testing data.
Evaluation Metrics
Performance was measured using Area Under the ROC Curve (AUC) and Average Precision (AP) across varying Signal-to-Noise Ratios (SNR), ranging from clean audio to high-noise environments (SNR -5).
Key Insights and Results
Robustness in High-Noise Scenarios
The adversarial models consistently outperformed baseline models in noisy conditions. At an SNR of -5 (where noise is significantly louder than the music), the proposed model achieved an AUC of 0.604 on Audioset-derived noise, compared to the baseline's 0.595. As the complexity and variety of noise increased, the performance gap between the proposed method and standard models widened, indicating superior generalization.
The Value of Unlabeled Data
A significant finding of this research is the utility of unlabeled noisy data. By including noisy samples without ground-truth tags during the adversarial phase, the model learned more robust representations. Interestingly, the researchers found that noise diversity is more critical than raw data volume; models trained on a smaller but more diverse set of noise types often outperformed those trained on larger, more homogenous datasets.
Staged Training Stability
The decision to pretrain the Domain Classifier separately (Step 2) proved vital. Initial experiments showed that training all components simultaneously led to instability and lower final accuracy. By allowing the DC to first "understand" the noise domain, the subsequent adversarial finetuning became more effective at forcing the FE to ignore those specific domain features.
Conclusion
This study demonstrates that Domain Adversarial Training, combined with a strategic three-step training process, significantly enhances the robustness of music auto-tagging models. By effectively utilizing unlabeled noisy data and focusing on noise-invariant feature extraction, the proposed framework offers a viable path toward more reliable music discovery tools in the complex acoustic environments of modern digital media.
Summary updated