Music genre perception is not a universal constant; it is deeply tied to language and culture. This research demonstrates that subjective perceptions can be accurately mapped across six different languages by combining distributional word embeddings with structured ontologies, significantly outperforming standard translation-based methods.
The Challenge of Cross-Lingual Genre Perception
In Music Information Retrieval (MIR), it is often assumed that a genre label like "funk" is semantically equivalent across cultures. However, a Western listener might associate funk with soul and jazz, while a Brazilian listener might associate it with baile funk, a derivative of rap.
Standard machine translation fails because it treats genre labels as static dictionary terms rather than culturally-bound concepts. Furthermore, creating a "parallel corpus"—where the same songs are manually labeled by experts in every language—is expensive and difficult to scale. This paper addresses unsupervised cross-lingual music genre annotation: predicting the correct genre tags for a song in a target language (e.g., Japanese) given its tags in a source language (e.g., English), without a pre-existing Rosetta Stone of labels.
Methodology: Embeddings and Ontologies
The researchers propose a framework that leverages two types of language-specific semantic representations:
- Distributed Concept Embeddings: The authors tested various multilingual models, including static vectors (fastText) and contextual models (mBERT, XLM, LASER). They found that Smooth Inverse Frequency (SIF) averaging on aligned fastText vectors provided a strong baseline.
- Music Genre Ontologies: Language-specific graphs were built from DBpedia (the structured version of Wikipedia) to capture genre relationships (e.g., "is a sub-genre of," "is a derivative of").
- Retrofitting (Rfit): This is the "secret sauce." Retrofitting is a graph-based optimization that "nudges" word embeddings so that concepts connected in an ontology (like "Rock" and "Punk") move closer together in the vector space. The authors provide a formal proof that their retrofitting objective is strictly convex, ensuring the algorithm always converges to the same optimal solution.
Dataset and Language Distribution
The authors introduced a new, domain-dependent cross-lingual corpus based on Wikipedia and DBpedia. It includes 63,246 music items (artists, albums, bands) across six languages: English (en), Dutch (nl), French (fr), Spanish (es), Czech (cs), and Japanese (ja).
As shown in the table above, the number of unique genres varies significantly by language, reflecting different levels of granularity in cultural categorization. The dataset captures how the same artist is tagged differently across language versions, providing a ground truth for cultural variance.
Key Results and Insights
Performance was measured using Macro-AUC (Area Under the ROC Curve).
- Translation Failure: Traditional translation (Google Translate) performed poorly, often struggling around 50-60% AUC. It cannot capture the fact that some cultures use more specific sub-genres while others use broad categories for the same music.
- Ontologies + Embeddings = Success: Combining distributional semantics with logical ontologies via retrofitting allows the model to learn cultural nuances. The best model consistently achieved scores in the 80-90% range across most language pairs.
- The "Rich-to-Poor" Mapping Effect: Mapping annotations from a "high-resource" language (like English, which has a massive, diverse genre vocabulary) to a "lower-resource" language is generally more accurate than the reverse.
Limitations and Future Directions
The imbalance in unique genres (e.g., 558 in English vs. 133 in Czech) reflects cultural reality but makes it harder to design perfectly symmetrical mapping functions. While the model aligns with Wikipedia's consensus, it remains subject to the inherent biases of Wikipedia contributors.
Future work involves applying this retrofitting approach to other subjective domains (like art or food) and integrating these cross-cultural models into music recommendation engines to provide more localized user experiences.
Summary updated