TL;DR
This paper introduces a novel method for music genre classification by learning shared vector representations for song lyrics and chord progressions. By treating lyrics and chords as two "languages" and embedding them in a common vector space, the authors demonstrate improved genre prediction, suggesting a deeper understanding of how these multimodal elements interact to shape our perception of music.
What Problem Does This Paper Address?
The paper tackles the challenge of understanding and classifying music, a complex, multimodal perceptual experience. While music genre is a key aspect of how we categorize and consume music, it's a subjective construct influenced by various elements. Existing approaches often focus on either lyrical content or musical structure (like chords) in isolation. The authors aim to bridge this gap by exploring how lyrics and chords interact and how this interaction can be leveraged for more accurate music genre classification. They pose questions like: how does musical structure affect our perception of lyrics, and vice versa? Can we better categorize a song with pop lyrics but R&B chords using a combined approach?
Methods
The core of the paper's methodology lies in adapting the "skip-gram" neural network architecture, commonly used in Natural Language Processing (NLP) for learning word embeddings, to a "bilingual" music context. Instead of just words, they learn embeddings for both lyrics and chord progressions simultaneously.
Here's a breakdown:
- Shared Vector Space: They create a joint vocabulary of words and chords and train a model to predict the context (surrounding words/chords) for a given word or chord. This is achieved by modifying the standard skip-gram objective function to include "cross-lingual" prediction tasks: predicting chords based on lyrics and vice versa.
- Bilingual Skip-Gram: This adaptation allows them to learn embeddings where lyrics and chords are represented in the same vector space, capturing their co-occurrence and interdependencies.
- Genre Classification: Once these shared embeddings are learned, they are used as features for a genre classification task. For each song, they average the embeddings of all its lyrical and chordal tokens to create a song-level feature vector. This vector is then fed into a k-Nearest Neighbors (k-NN) classifier.
Datasets
- Ukutabs: The primary dataset used for training the embeddings and for the genre classification task. This website provides a large archive of popular songs with aligned chordal and lyrical passages. The authors curated a dataset of 5,304 songs from Ukutabs, resulting in 190,165 musical segments.
- Billboard Charts: Used as the ground truth for genre labels. The authors collected song titles from Latin, Rock, R&B/Hip-Hop, and Pop Billboard charts over the last 20 years and matched them to their Ukutabs dataset, resulting in 850 unique songs for evaluation.
Key Insights and Ideas
- Lyrics and Chords are Intertwined: The most significant insight is that learning shared representations for lyrics and chords demonstrably improves music genre classification. This suggests that the interplay between lyrical content and harmonic structure is a crucial factor in how we perceive and categorize music.
- Multimodal Approach is Key: The "Chords & Lyrics" model, which leverages joint embeddings, significantly outperforms models that use only chord embeddings or only lyric embeddings. This strongly supports the idea that a multimodal approach is more effective for understanding music perception.
- NLP Techniques for Music: The paper highlights the power of adapting successful NLP techniques (like word embeddings) to the domain of music, opening up new avenues for research in music information retrieval and computational musicology.
- Genre as a Perceptual Construct: By using genre classification as a proxy for music perception, the study underscores the subjective yet classifiable nature of musical experiences.
Key Results
The "Chords & Lyrics" model, using joint embeddings, achieved the best performance across several metrics for multi-label genre classification:
- Exact Match Ratio (EMR): 20.4%
- Label Accuracy: 37.9%
- Label-based, Micro-averaged F1-score: 0.428
These results significantly outperformed baseline models (like predicting the most common genre) and monolingual embedding models (Chords Only, Lyrics Only). For instance, the "Chords & Lyrics" model achieved a label accuracy of 37.9%, compared to 18.5% for "Lyrics Only" and 16.5% for "Chords Only."
Limitations and Future Work
- Dataset Bias: The Ukutabs dataset has a bias towards music playable by ukulele musicians, which might limit the generalizability of the findings.
- Chord Caster Accuracy: The automated chord caster used to standardize chord types was untested and might have introduced inaccuracies in up to 4.1% of chords.
- Audio Features: The current study relies solely on symbolic representations (lyrics and chords). The authors suggest future work should compare their symbolic approach with models that incorporate auditory features (like rhythm and harmony) to gain a deeper understanding of music perception.
- Weighting Objective Function: The paper mentions that different weight coefficients in the objective function could be explored to emphasize certain aspects (mono-lingual vs. cross-lingual) of the embeddings.
- Advanced Classifiers: While k-NN was used for its simplicity and speed, exploring more advanced multi-label classification models (e.g., RAKEL) could potentially yield even better results.
- Key Estimation: The simple key estimation method used was effective but requires further investigation for broader generalization.
Summary updated