Modeling the Music Genre Perception across Language-Bound Cultures

Elena V. Epure, Guillaume Salha, Manuel Moussallam, Romain Hennequin

TL;DR

This paper tackles the challenge of accurately annotating music genres across different cultures and languages, recognizing that genre perception isn't universal and can't be solved by simple translation. They demonstrate that by combining language-specific semantic representations like ontologies and word embeddings, they can achieve high accuracy in cross-lingual music genre annotation, even without parallel music item data.

What Problem Does This Paper Address?

Music genres are a fundamental way we categorize and understand music, but their perception is not uniform. A Westerner might associate "funk" with soul and jazz, while a Brazilian might link it to "baile funk," a type of rap. This cultural subjectivity means that simply translating genre labels from one language to another is insufficient for accurate cross-lingual music genre annotation. The paper addresses the gap in understanding and modeling these culturally-specific variations in music genre perception, aiming to build systems that can accurately annotate music across diverse linguistic and cultural backgrounds without relying on large, parallel datasets of music items and their genre tags.

Methods

The core of the paper's methodology lies in leveraging language-specific semantic representations to bridge cultural divides in music genre perception. They explore two main types:

  • Ontologies: These are structured knowledge bases that represent music genres and their relationships (e.g., "hip hop" is a parent genre of "rap west coast"). The authors utilize Wikipedia as a source for these ontologies, acknowledging that even within Wikipedia, genre definitions and relationships can reflect cultural nuances.
  • Distributed Representations (Embeddings): These are vector representations of words or phrases that capture semantic meaning based on their context. The paper investigates various multilingual pre-trained embedding models, including:
  • Static Word Embeddings: Specifically, multilingual fastText.
  • Contextual Word Embeddings: Multilingual BERT (mBERT) and XLM.
  • Universal Language-Agnostic Sentence Embeddings: LASER.

A key novel aspect is the retrofitting technique. This method refines existing word embeddings by incorporating information from ontologies. They apply this in two ways:

  1. Monolingual Retrofitting: Refining language-specific embeddings using language-specific ontologies.
  2. Cross-lingual Retrofitting (from scratch): Learning multilingual embeddings by partially aligning ontologies across languages and then applying retrofitting.

Datasets

The primary dataset used in this work is derived from Wikipedia. The authors crawled Wikipedia-based music genre ontologies for six languages: English (en), Dutch (nl), French (fr), Spanish (es), Czech (cs), and Japanese (ja).

They also constructed a new, domain-dependent cross-lingual corpus for benchmarking. This corpus consists of music items (artists and albums) annotated with music genres, collected from DBpedia. They specifically targeted items annotated in at least two of the six languages and filtered for genres appearing at least 15 times in the corpus, resulting in a dataset of 63,246 music items. This corpus is notable for its focus on music genres and its cross-lingual nature, aiming to capture cultural differences in genre perception.

Key Insights and Ideas

  • Translation is Not Enough: The most striking insight is that simple translation of genre labels fails to capture the nuanced, culturally-dependent perception of music genres. This highlights the need for models that understand the underlying semantic and cultural context.
  • Ontologies and Embeddings are Powerful Together: Combining language-specific ontologies with distributed representations (embeddings) significantly boosts the accuracy of cross-lingual music genre annotation. This synergy suggests that structured knowledge and distributional semantics are complementary.
  • Retrofitting for Domain Adaptation: The retrofitting technique proves highly effective in adapting general-purpose language embeddings to the specific domain of music genres, leading to improved annotation performance.
  • Smooth Inverse Frequency Averaging (sif) is a Strong Baseline: The paper highlights that a simple method like sif averaging of aligned word embeddings can serve as a surprisingly strong baseline for cross-lingual tasks, outperforming more complex approaches in some cases.
  • Cultural Nuances in Wikipedia: The study leverages the fact that Wikipedia's multilingual contributions, while aiming for neutrality, can still reflect cultural differences in how topics, including music genres, are described and categorized.

Key Results

The paper reports significant improvements in cross-lingual music genre annotation accuracy using their proposed methods.

  • Translation Baselines: Standard translation methods (Google Translate and DBpedia's SameAs relation) achieved lower Macro-AUC scores compared to semantic representation-based methods. For example, translation baselines often scored in the 50s-70s%, while semantic methods reached the high 80s-90s%.
  • Distributed Representations:
    • Averaging static multilingual word embeddings with the sif method (specifically aligned fastText vectors) performed very well, often achieving the best results for 5 out of 6 source languages.
  • Contextual embeddings from XLM (XLMctxt) outperformed mBERTctxt.
  • LASER generally underperformed compared to other methods, except for Japanese.
  • Retrofitted Embeddings:
    • Retrofiting fastText embeddings to monolingual ontologies significantly improved music genre annotation modeling across all language pairs, pushing scores into the low to mid-90s%.
  • Learning embeddings from scratch using retrofitting with partially aligned ontologies also yielded strong results, particularly when English or Spanish were the source languages.
  • Language-Specific Performance: The lowest scores were observed when Japanese was the source language, potentially due to the quality of embedding models for Japanese or the completeness of its music genre ontology.

Limitations and Future Work

  • Inadequately Modeled Gaps: While the methods show high accuracy, the authors acknowledge that larger gaps in music genre perception might still be inadequately modeled. For instance, a specific jazz album tagged in Spanish might not be correctly predicted in French.
  • Unbalanced Genre Data: The study observed that languages with more music genre tags (like English and Spanish) generally performed better as sources. This imbalance might be a cultural consequence, and future work could explore mapping functions that leverage this.
  • Future Work:
    • Investigating better mapping functions to handle unbalanced music genre data.
  • Exploring music genre relations as paraphrasing forms, inspired by sentence embedding learning.
  • Thoroughly evaluating the developed models in downstream music retrieval and recommendation tasks.
  • Further probing the effectiveness of language-specific concept representations in other domains.

Summary updated

Abstract

The music genre perception expressed through human annotations of artists or albums varies significantly across language-bound cultures. These variations cannot be modeled as mere translations since we also need to account for cultural differences in the music genre perception. In this work, we study the feasibility of obtaining relevant cross-lingual, culture-specific music genre annotations based only on language-specific semantic representations, namely distributed concept embeddings and ontologies. Our study, focused on six languages, shows that unsupervised cross-lingual music genre annotation is feasible with high accuracy, especially when combining both types of representations. This approach of studying music genres is the most extensive to date and has many implications in musicology and music information retrieval. Besides, we introduce a new, domain-dependent cross-lingual corpus to benchmark state of the art multilingual pre-trained embedding models.
BibTeX
@inproceedings{epure2020modeling,
  author = {Elena V. Epure and Guillaume Salha and Manuel Moussallam and Romain Hennequin},
  title = {{Modeling the Music Genre Perception across Language-Bound Cultures}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.386},
}
Figure