Perception-Inspired Graph Convolution for Music Understanding Tasks

Emmanouil Karystinaios, Francesco Foscarin, Gerhard WidmerIJCAI 2024

Abstract

Symbolic music representation poses unique challenges for machine learning, as musical meaning often resides in the relative relationships between notes rather than absolute values. We present MusGConv, a novel graph convolutional block designed to incorporate human perceptual principles—such as relative pitch and rhythm—directly into the architecture of Graph Neural Networks (GNNs). By explicitly modeling note relationships as edge features, MusGConv achieves state-of-the-art performance in tasks like voice separation and composer identification without increasing computational costs.

Introduction

In Music Information Retrieval (MIR), symbolic music (e.g., MIDI or digital scores) is frequently treated as a sequence of tokens or a two-dimensional "piano roll" image. However, music is inherently structured through overlapping notes and varying durations where the "meaning" of a note is defined by its context—specifically its distance from other notes (intervals).

While GNNs are naturally suited for this graph-like structure, standard architectures often fail to capture fundamental musical truths. For example, humans perceive a melody identically regardless of its key (transposition invariance). Generic GNNs typically require extensive data augmentation to learn this concept. MusGConv addresses this by encoding perceptual principles directly into the graph convolution process.

High-level pipeline of the MusGConv framework, from musical score to task-specific outputs.

Methodology

The core of the proposed approach is the MusGConv block, which replaces standard message-passing layers in a GNN to better handle musical data.

Graph Construction and Edge Features

Notes are represented as nodes, with edges defined by four temporal relationships: onset (simultaneous start), during (overlap), follow (sequential), and silence (gap). Unlike standard GNNs that focus primarily on node attributes, MusGConv calculates explicit features for every edge:

  1. Temporal and Pitch Distances: Absolute differences in onset time, duration, and MIDI pitch.
  2. Pitch-Class Interval (PCInt): A specialized feature capturing "chroma" distance (e.g., the interval between C and G regardless of octave).

The MusGConv Block

The model utilizes a dedicated Multi-Layer Perceptron (MLP) to process these edge features, which are then concatenated with node features during message passing. This allows the network to prioritize relative information (intervals and rhythmic ratios) alongside absolute information (MIDI pitch).

Detailed architecture of the MusGConv block, illustrating the integration of edge features into the message-passing phase.

Experimental Results

MusGConv was evaluated across four distinct MIR tasks: voice separation, composer classification, Roman Numeral Analysis (RNA), and cadence detection.

Performance Gains

The model demonstrated significant improvements over generic GNN baselines:

  • Voice Separation: Achieved an F1 score of 0.8436, surpassing the previous SOTA of 0.8111.
  • Composer Classification: Accuracy increased from 0.4288 (baseline) to 0.5233.
  • Cadence Detection: Improved F1 scores from 0.4065 to 0.4295.

Efficiency and Invariance

A key advantage of MusGConv is its architectural transposition invariance. By encoding pitch distances on edges, the model recognizes patterns (like a Major 3rd) regardless of the absolute key. This is more efficient than data augmentation. Furthermore, execution time remained virtually identical to baseline models, providing a "free" performance upgrade.

Discussion and Conclusion

While MusGConv excelled in most tasks, it did not significantly improve Roman Numeral Analysis (RNA). This is likely because existing RNA models are already highly complex and utilize heavy data augmentation, which may have already taught the baseline the relative pitch concepts that MusGConv provides architecturally.

In conclusion, MusGConv proves that incorporating domain-specific perceptual knowledge into GNN architectures significantly boosts performance in music understanding tasks. Future work will explore its application to diverse musical genres and the integration of more complex human expectation models.

Summary updated

BibTeX
@inproceedings{karystinaios2024perception,
  author = {Emmanouil Karystinaios and Francesco Foscarin and Gerhard Widmer},
  title = {{Perception-Inspired Graph Convolution for Music Understanding Tasks}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure