TL;DR
This paper presents a novel multimodal computational analysis of protest songs, using both lyrics and audio to identify distinguishing features. They introduce a new dataset and demonstrate that protest songs are acoustically and linguistically distinct, with lyrics playing a crucial role in conveying dissent through general linguistic markers.
What Problem Does This Paper Address?
Music has long been a powerful tool for political expression and social change, with protest songs serving as anthems for movements worldwide. However, despite their cultural significance, the specific linguistic and acoustic characteristics that define protest music remain underexplored. Existing research often relies on manual analysis, limiting the scale and depth of investigation. This paper aims to bridge this gap by computationally analyzing a large dataset of protest and non-protest songs to uncover the systematic differences in their lyrics and audio.
Methods
The authors employ a multimodal approach, analyzing both textual (lyrics) and audio components of songs.
- Text-based Analysis: They use various transformer-based language models (XLM-ROBERTa, RoBERTa, DistilBERT) to generate embeddings from lyrics. Additionally, they extract interpretable linguistic features like sentiment score, rhyme density, lexical diversity, and n-gram repetition. These features are then used to train traditional classifiers.
- Audio-based Analysis: Pretrained audio models (CLAP, HuBERT, Wav2Vec2) are used to extract deep audio representations. They also extract low-level audio features like spectral flux, shimmer, and MFCCs.
- Source Separation: To understand the contribution of vocals versus accompaniment, they use Spleeter to separate audio tracks into two stems. Each stem is then classified independently, and a controlled mixing experiment is conducted.
- Human Annotation: A subset of songs was annotated by humans on musical attributes like speed, energy, danceability, acousticness, instrumentation, ornamentation, and disjunctness to validate computational findings.
Datasets
The study utilizes a newly curated multimodal dataset:
- Protest Songs: 446 songs sourced from a list compiled from Wikipedia (originally 459 tracks).
- Non-Protest Songs: 370 songs selected using GPT-4 inference to ensure they are not associated with social or political movements, matched for time period and genre diversity.
The dataset includes full lyrics, 30-second audio excerpts, and source-separated vocal/accompaniment tracks.
Key Insights and Ideas
- Linguistic Markers of Dissent: Protest songs are not just about specific political themes but rely on broader linguistic patterns. Features like higher repetition, lexical diversity, and sentiment polarity are key differentiators.
- Vocal Dominance, but Accompaniment Matters: While vocals are crucial for conveying protest intent, the accompaniment also plays a significant role. Interestingly, in intermixing experiments, accompaniment sometimes contributed more to perceived protestiveness than vocals.
- Music-Specific Audio Features are Key: For audio analysis, models trained on music-specific features (like CLAP) outperform those trained on general speech or acoustic cues. This highlights the importance of timbre, rhythm, and expressive style in protest music.
- Multimodal Signature: Protest music conveys its message through a combination of linguistic urgency and expressive musicality, creating a unique, identifiable signature across both text and sound.
Key Results
- Textual Classification: XLM-ROBERTa achieved the highest F1-score of 91.10% for protest song classification based on lyrics. Interpretable linguistic features also performed well, outperforming specialized lyric models.
- Audio Classification: The CLAP model achieved the highest F1-score of 90.62% for audio classification. Models trained on music-specific features (like spectral flux and repetition) also performed strongly.
- Source Separation: Both CLAP and HuBERT showed higher F1-scores for vocal classification than accompaniment. However, when songs were mixed, accompaniment was more often flagged as protest content, suggesting models might sometimes misattribute signals.
- Human Annotation: Statistically significant differences were found in features like speed, energy, and spectral contrast between protest and non-protest songs, supporting computational findings.
Limitations and Future Work
- Audio Availability: Limitations in publicly accessible audio for some protest songs.
- Genre Balance: While an effort was made, genre remains a potential confounding variable. Future work should explicitly control for genre.
- Model Biases: The intermixing experiment suggests potential biases in how models interpret musical features, leading to misattribution of protest signals.
- Future Directions:
- Expand the dataset to include non-Western protest traditions and incorporate temporal metadata for diachronic analysis.
- Develop joint lyric-audio models with cross-modal attention.
- Investigate the impact of social media on protest music creation and perception.
- Incorporate listener surveys and focus groups for deeper human-centered evaluation.
- Involve trained musicians in annotation for more complex musical features.
Summary updated