MediaEval 2020 Emotion and Theme Recognition in Music Task: Loss Function Approaches for Multi-label Music Tagging

Dillon Knox, Timothy Greer, Benjamin Ma, Emily Kuo, Krishna Somandepalli, Shrikanth NarayananMediaEval 2020 · OpenReview

Introduction

Automatic music tagging—assigning labels like "melancholic," "epic," or "driving"—is a notoriously difficult sparse multi-label problem. In a typical dataset, a single track might only have two or three relevant tags out of hundreds, creating a massive "negative label" problem. Furthermore, music data is "long-tailed": a few tags (the "head") appear constantly, while most tags (the "tail") appear very rarely. Standard training techniques often result in models that perform well on common tags but fail on rare ones. Researchers from USC’s SAIL lab aimed to address this by implementing loss functions that force models to pay more attention to these infrequent, "hard-to-learn" labels.

Methodology

The authors built upon a state-of-the-art short-chunk CNN with residual connections, introducing two primary modifications:

  1. Increased Receptive Field: They modified kernel sizes in the final convolutional layers to increase the temporal window from 3.69 seconds to 4.6 seconds, providing more context for high-level descriptors like emotion.
  2. Advanced Loss Functions: Instead of standard Binary Cross-Entropy (BCE), they experimented with Focal Loss, Class-Balanced Loss, and Distribution-Balanced Loss. These functions are designed to reduce the influence of easy examples or weight classes based on their frequency and co-occurrence.

The models were trained using the MTG-Jamendo dataset, supplemented by Music4All for expanded training data, and pre-trained on the Million Song Dataset (MSD). An ensemble model was created by averaging predictions from models using these different loss functions.

Results and Discussion

The authors' approach outperformed several baselines and previous top models. The final ensemble reached a PR-AUC of 0.161 and an ROC-AUC of 0.781.

Overall performance metrics for different loss functions and baselines.

A key finding was that changing the loss function provided a more significant performance boost than traditional resampling. Specifically, Focal Loss-based methods improved performance on "middle" and "tail" classes (those with fewer than 550 training samples).

Performance comparison across Head, Middle, and Tail class distributions.

As shown in the table above, while there is a slight penalty to "head" classes, the overall system becomes more robust. On "tail" classes, Focal Loss achieved a PR-AUC of 0.113, a significant improvement over BCE’s 0.101. Other insights included the observation that increasing the temporal receptive field directly improved recognition of emotions, and that "mixup" augmentation actually decreased performance when paired with Focal Loss variants.

Conclusion

The researchers demonstrated that advanced loss functions are effective for addressing the long-tail problem in multi-label music tagging. While common techniques like class-aware sampling led to performance degradation in this specific architecture, the ensemble of loss-optimized models provided a robust solution. Future work will explore the generalization of these strategies to other tagging tasks such as genre or instrument detection.

Summary updated

BibTeX
@inproceedings{knox2020mediaeval,
  author = {Dillon Knox and Timothy Greer and Benjamin Ma and Emily Kuo and Krishna Somandepalli and Shrikanth Narayanan},
  title = {{MediaEval 2020 Emotion and Theme Recognition in Music Task: Loss Function Approaches for Multi-label Music Tagging}},
  booktitle = {MediaEval},
  year = {2020},
  url = {https://openreview.net/forum?id=gy-A8yx_9Sg},
}
Figure