Towards Cross-Cultural Analysis using Music Information Dynamics

Shlomo Dubnov, Kevin Huang, Cheng-i WangarXiv 2021 · OpenReview

Abstract

Music is comprehended both hierarchically, from sonic events to melodies, and sequentially, through repetition and variation. Different cultures establish unique aesthetics by employing varying style conventions across these two dimensions. We propose a framework to quantitatively compare music from different cultures using a Music Information Dynamics model, specifically a Variable Markov Oracle (VMO) extended with variational representation learning. A variational autoencoder (VAE) maps audio fragments into a latent representation, which is then processed by the VMO. The VMO learns a clustering of the latent representation via a threshold that maximizes the Information Rate (IR). This approach allows for the characterization of musical signals at varying levels of acoustic sensibility. Our findings indicate that sensibility to subtle acoustic changes is higher in East-Asian musical traditions, while Western works exhibit longer motivic structures at higher thresholds of difference in the latent space.

Introduction

Cross-cultural music analysis has traditionally relied on hand-crafted audio features like Mel-frequency Cepstral Coefficients (MFCCs) or Chroma. However, these features often carry inherent cultural biases. Chroma features, for instance, aggregate sound into 12 bins corresponding to the Western twelve-tone equal temperament system, potentially overlooking microtonal or timbral nuances essential to non-Western music.

To address this, we developed a culturally agnostic framework that captures both the hierarchical "sound units" and their sequential temporal structures. By combining unsupervised deep learning with information theory, we can objectively compare the aesthetic differences between musical traditions without relying on pre-defined musicological categories.

Methodology

The framework operates in two stages:

  1. Representation Learning (Hierarchical): A Convolutional Variational Autoencoder (VAE) compresses time-frequency spectrograms into a latent representation ($Z$). As an unsupervised model trained to reconstruct audio, the VAE acts as an unbiased feature extractor for basic "sound units."
  2. Temporal Structure Discovery (Sequential): The latent sequence is analyzed by a Variable Markov Oracle (VMO). The VMO clusters these representations using a distance threshold ($\theta$) that maximizes the Information Rate (IR)—a metric representing the optimal balance between predictability and variation.

By plotting IR against different thresholds, we create an "IR profile" that characterizes the information content of a signal at different levels of acoustic detail. The VMO also extracts repeated sub-strings, or motifs, based on the optimal threshold.

Results and Discussion

Information Rate Profiles

The IR profiles reveal distinct cultural differences in acoustic sensibility. East-Asian music (Shakuhachi and Dongxiao) exhibits higher IR peaks at lower distance thresholds. This suggests that structural information in these traditions is embedded in subtle, micro-level timbral and breath variations. In contrast, Western music (Bach and Telemann) peaks at higher thresholds, indicating that its structure is primarily built on coarser, tonal-level changes, such as distinct pitches.

Summed Information Rate (IR) profiles as a function of threshold value for different musical works and feature extractors.

Furthermore, the framework proved sensitive enough to detect instrument-specific mechanics. For example, the Telemann piece played on a wooden recorder showed a secondary IR peak at a lower threshold compared to the modern metal flute version, successfully capturing the finer acoustic details inherent to the recorder.

Motif Length and Sequential Structure

Analysis of the extracted motifs further differentiates the musical traditions. Western pieces yielded significantly longer motifs, aligning with a composition style driven by extended tonal progressions and rigid phrasing. East-Asian pieces produced much shorter motifs, reflecting an aesthetic focused on the expressive inflection of individual sound units rather than long-form melodic development.

Violin plots showing the distribution of extracted pattern lengths across different musical traditions and features.

Statistical tests (ANOVA, $p < 10^{-5}$) confirmed a highly significant difference in mean motif lengths between the two cultures. This quantitative evidence supports the qualitative observation that Western music emphasizes sequential development, while East-Asian music emphasizes hierarchical, timbral detail.

Conclusion

This study demonstrates that a profile of information contents, analyzed as a function of acoustic detail, can serve as a powerful cultural characteristic. The VAE-VMO framework successfully bypassed the cultural biases of hand-crafted features while capturing meaningful musical structures.

While the current dataset is limited to solo flute recordings, future work will involve scaling this approach to larger corpora and diverse instrument families. We also intend to integrate the VAE and VMO components into a single, end-to-end model for more holistic music information dynamics analysis.

Summary updated

Abstract

A music piece is both comprehended hierarchically, from sonic events to melodies, and sequentially, in the form of repetition and variation. Music from different cultures establish different aesthetics by having different style conventions on these two aspects. We propose a framework that could be used to quantitatively compare music from different cultures by looking at these two aspects. The framework is based on an Music Information Dynamics model, a Variable Markov Oracle (VMO), and is extended with a variational representation learning of audio. A variational autoencoder (VAE) is trained to map audio fragments into a latent representation. The latent representation is fed into a VMO. The VMO then learns a clustering of the latent representation via a threshold that maximizes the information rate of the quantized latent representation sequence. This threshold effectively controls the sensibility of the predictive step to acoustic changes, which determines the framework's ability to track repetitions on longer time scales. This approach allows characterization of the overall information contents of a musical signal at each level of acoustic sensibility. Our findings under this framework show that sensibility to subtle acoustic changes is higher for East-Asian musical traditions, while the Western works exhibit longer motivic structures at higher thresholds of differences in the latent space. This suggests that a profile of information contents, analyzed as a function of the level of acoustic detail can serve as a possible cultural characteristic.
BibTeX
@misc{dubnov2021towards,
  author = {Shlomo Dubnov and Kevin Huang and Cheng-i Wang},
  title = {{Towards Cross-Cultural Analysis using Music Information Dynamics}},
  year = {2021},
  eprint = {2111.12588},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.12588},
}
Figure