Graph Representations of Hierarchical Musical Structure

This direction is merged into another direction: hierarchical music representation, heterogeneous graph music representation. Its last version stays readable and citable here.

Version 1 of 117 papers27 conceptsfrozen 2026-10-04

Traditional machine learning treats music as a sequence—a linear string of notes or audio frames. But to a musician, a score is less like a sentence and more like a complex web. A single note might simultaneously belong to a melodic line, a harmonic chord, and a rhythmic pattern, while also serving as a "prolongation" of a structural goal several measures away.

Recent research has shifted toward representing music as multi-relational graphs. By moving beyond the sequence, researchers are able to capture the deep, hierarchical structures that give music its coherence, enabling models to analyze and generate compositions that feel fundamentally more "human."

Music as a Multi-Relational Web

In a graph-based representation, notes are treated as nodes, and the relationships between them—temporal, harmonic, or structural—are treated as edges. This allows a model to "see" connections that are invisible to sequential models like Transformers.

For instance, Emmanouil Karystinaios, Francesco Foscarin, and Gerhard Widmer introduced MusGConv, a graph convolutional framework that explicitly models perceptual relationships. Instead of just knowing that Note B follows Note A, the model understands their relative pitch and rhythmic distance as edge features. This spatial awareness allows the system to excel at tasks like voice separation, where the model must distinguish between multiple overlapping melodic lines.

This approach extends to polyphonic and multitrack music. Emanuele Cosenza and colleagues demonstrated that by modeling multitrack interactions as edges, a system can ensure that a melody in the violin track remains harmonically synchronized with the chords in the piano track.

Uncovering the Deep Structure

One of the most significant breakthroughs in this area is the automation of Schenkerian analysis, a music theory method that strips away surface-level "ornaments" to reveal a piece's fundamental structural skeleton.

Early work by Phillip B. Kirlin and David D. Jensen modeled these hierarchies using maximal outerplanar graphs, where triangles represent the relationship between a parent structure and its decorative child nodes.

More recently, the AutoSchA framework, developed by Stephen Ni-Hahn, Cynthia Rudin, and Simon Mak, uses a "node isolation" pooling technique. By treating the score as a multi-relational graph, the model can iteratively "collapse" less important notes into their structural parents. This mimics the way a human analyst identifies how a flourish of sixteenth notes eventually resolves into a single, stable structural tone.

From Analysis to Generation

The power of graph representations isn't limited to understanding existing music; it is also being used to create new music with better long-term coherence.

Standard AI generators often suffer from "drift," where a song starts in one key or style but loses its way after thirty seconds. By using graph-based architectures, models can maintain a "global" view of the piece. Jinhua Liang and Huan Zhang use multi-layered graphs where nodes represent entire phrases or chords, ensuring that the high-level "story arc" of a pop song (intro, verse, chorus) remains intact.

Taking this a step further, the ProGress framework combines graph diffusion with Schenkerian theory. Instead of predicting the next note in a sequence, the model generates a hierarchical graph. This ensures that every note generated is anchored to a structural foundation, resulting in compositions that Stephen Ni-Hahn and his team found to be more interpretable and "human-sounding" than those from massive black-box sequential models.

Beyond the Score: Audio Event Graphs

The intuition that "structure is a graph" is even moving into raw audio synthesis. In text-to-audio generation, models often struggle with precise timing—for example, failing to put a "doorbell" exactly after a "dog bark."

Yan Liu and the creators of DegDiT address this by representing audio scenes as dynamic event graphs. In these graphs, nodes are sound events and edges define temporal relationships like "overlapping" or "containing." By guiding a diffusion transformer with this graph, the model gains a structural blueprint that flat text descriptions cannot provide, allowing for surgical control over the timing of a generated acoustic scene.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_graph_representations_of_hierarchical_mu_d4b44aa4,
  title = {Graph Representations of Hierarchical Musical Structure},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/graph-representations-of-hierarchical-musical-structure/rdr_d4b44aa4bc3f46cfaa9e27589802e346},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_graph_representations_of_hierarchical_mu_d4b44aa4_v1,
  title = {Graph Representations of Hierarchical Musical Structure},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/graph-representations-of-hierarchical-musical-structure/rdr_d4b44aa4bc3f46cfaa9e27589802e346/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}