Latent evolutionary signatures: a general framework for analysing music and cultural evolution

Jonathan Warrell, Leonidas Salichos, Michael Gancz, Mark GersteinJournal of The Royal Society Interface 2024 · DOI

TL;DR

This paper introduces a novel framework for analyzing cultural evolution, drawing parallels with biological evolution. They propose a deep generative model that extracts "latent evolutionary signatures" from music, akin to mutational signatures in genomics, to understand underlying evolutionary processes and units of cultural change. These learned representations significantly outperform standard methods in predicting song period and genre.

What Problem Does This Paper Address?

Cultural evolution, particularly in domains like music, has long been observed to share similarities with biological evolution. However, a key challenge has been identifying the fundamental "units" of cultural transmission and the specific evolutionary processes that drive change. Unlike biological evolution, where genes serve as clear units and mutations as processes, cultural equivalents have remained elusive, especially in music. Previous approaches often focused on predefined features or statistical analyses without explicitly modeling the evolutionary dynamics. This paper aims to bridge this gap by developing a general framework to simultaneously identify these underlying cultural units and their evolutionary trajectories, using music as a case study.

Methods

The core of the paper is a novel deep generative model, framed as a variational autoencoder (VAE) with an "evolutionary prior." This prior is implemented using an energy-based model that links songs based on their temporal proximity (or other defined evolutionary relationships), encouraging similar songs in "evolutionary space" to have similar latent representations.

  1. Feature Extraction: The model first extracts harmonic features from songs, specifically focusing on chord transitions (k-mers) and formal repetition schemes. These are represented as vectors.
  2. Latent Signature Learning: A deep neural network is trained to reconstruct these features from a set of binary latent codes, where each code represents an "evolutionary signature." The model is optimized using a variational Bayes expectation-maximization (VBEM) approach.
  3. Evolutionary Prior: The key innovation is the energy-based prior that penalizes large Hamming distances between latent codes of temporally close songs, effectively embedding an evolutionary constraint into the learning process. This encourages the model to learn signatures that evolve gradually over time.
  4. Prediction Tasks: The learned latent representations are then evaluated on their ability to predict song period (chronological order) and genre using kernel regression.

Datasets

The primary dataset used is the McGill Billboard dataset, containing approximately 730 songs from 533 artists spanning 1958-1991, categorized into 64 genres. This dataset was split into training (584 songs) and testing (146 songs) sets, with the test set formed by taking every fifth song chronologically to maintain temporal order.

Key Insights and Ideas

  • Music as an Evolutionary System: The paper strongly advocates for viewing music not just as art but as a system undergoing evolutionary processes, analogous to biological evolution.
  • Latent Signatures as Cultural Units: The concept of "latent evolutionary signatures" offers a powerful way to uncover underlying, unobserved units of musical change, similar to how mutational signatures reveal processes in genomics.
  • Evolutionary Prior is Crucial: Incorporating an explicit evolutionary prior (based on temporal proximity) into the generative model significantly improves its ability to capture meaningful patterns compared to standard VAEs. This suggests that temporal dynamics are fundamental to understanding cultural evolution.
  • Interpretable Signatures: The learned signatures, while latent, can be interpreted by examining the features they prioritize, offering insights into evolving harmonic syntax and formal structures.
  • Predictive Power: The evolutionary representations are demonstrably more informative for downstream tasks like period and genre prediction, highlighting their utility beyond mere pattern discovery.

Key Results

  • Period and Genre Prediction: The evolutionary signature models achieved significantly better performance (measured by Pearson correlation) on period and genre prediction tasks compared to standard VAE models. For period prediction, the best evolutionary model achieved an r of 0.275, compared to 0.220 for raw features and 0.217 for a standard VAE. For genre prediction, the best evolutionary model achieved an r of 0.243, compared to 0.217 for raw features and 0.18 for a standard VAE.
  • Signature Dynamics: The learned signatures showed clear temporal trends, with some increasing or decreasing in activation over time, and others exhibiting oscillatory patterns, suggesting distinct evolutionary trajectories.
  • Feature Prioritization: The models identified signatures that emphasized specific harmonic features (e.g., diatonic chords, bVI/bVII chords) and formal structures (e.g., repetition schemes), providing musicological insights. For instance, one signature showed a decrease in variance over time, suggesting a "homogenization" trend in 4-mer sequences.
  • Combination of Features: Combining harmonic and formal features in the model led to optimal performance, suggesting that both aspects are crucial for a comprehensive understanding of musical evolution.

Limitations and Future Work

  • Oversimplified Model: The authors acknowledge that their model is an "initial attempt" and uses a simplified memetic-based evolutionary architecture with discrete binary signatures and a simple mutation rate.
  • Fitness and Adaptation: The current model primarily focuses on neutral evolutionary processes. Future work could explore incorporating fitness parameters to model adaptive evolution, where the number of offspring depends on heritable phenotypes.
  • Continuous Latent Space: The latent signatures are currently binary. Extending this to a continuous latent space could better reflect psychological biases or more complex variation mechanisms.
  • Learning the Graph: The evolutionary graph (G) was predefined based on temporal proximity. Future work could involve learning this graph jointly with other model parameters to discover more nuanced influence relationships.
  • Broader Domains: The framework is generalizable to other cultural domains (e.g., language, visual arts) and cross-domain analysis.
  • Novelty and Creativity: Incorporating models of creativity and exploring how novelty arises within this evolutionary framework is a promising direction.
  • Recommender Systems: The learned latent representations could be valuable for developing advanced recommender systems.

Summary updated

Abstract

Cultural processes of change bear many resemblances to biological evolution. The underlying units of non-biological evolution have, however, remained elusive, especially in the domain of music. Here, we introduce a general framework to jointly identify underlying units and their associated evolutionary processes. We model musical styles and principles of organization in dimensions such as harmony and form as following an evolutionary process. Furthermore, we propose that such processes can be identified by extracting latent evolutionary signatures from musical corpora, analogously to identifying mutational signatures in genomics. These signatures provide a latent embedding for each song or musical piece. We develop a deep generative architecture for our model, which can be viewed as a type of variational autoencoder with an evolutionary prior constraining the latent space; specifically, the embeddings for each song are tied together via an energy-based prior, which encourages songs close in evolutionary space to share similar representations. As illustration, we analyse songs from the McGill Billboard dataset. We find frequent chord transitions and formal repetition schemes and identify latent evolutionary signatures related to these features. Finally, we show that the latent evolutionary representations learned by our model outperform non-evolutionary representations in such tasks as period and genre prediction.
BibTeX
@article{warrell2024latent,
  author = {Jonathan Warrell and Leonidas Salichos and Michael Gancz and Mark Gerstein},
  title = {{Latent evolutionary signatures: a general framework for analysing music and cultural evolution}},
  journal = {Journal of The Royal Society Interface},
  year = {2024},
  doi = {10.1098/rsif.2023.0647},
}
Figure