MELONET I: Neural Nets for Inventing Baroque-Style Chorale Variations

Dominik HörnelNeurIPS 1997

Introduction

In the mid-1990s, neural networks were already demonstrating promise in algorithmic composition, but they suffered from a significant "memory" problem. While models could easily learn local transitions—predicting which note logically follows another—they struggled with global coherence. Music is not merely a string of notes; it is a nested structure of motifs, phrases, and harmonies occurring across different time scales. Earlier models often produced "aimless" melodies because they lacked a mechanism to plan ahead.

MELONET I addresses this gap by mimicking how a human organist improvises: establishing a "rough idea" of a melodic shape (a motif) before committing to individual notes. The result is a hierarchical neural network system designed to compose Baroque-style melodic variations that experts find comparable to human organist improvisations.

The Multi-Scale Architecture

To achieve global coherence, the researchers decomposed the composition task into distinct sub-tasks using a multi-scale approach.

MELONET I Architecture
  1. Harmonization (HARMONET): Before melodic variations begin, an existing system called HARMONET takes a simple melody and invents a four-part chorale harmonization in the style of J.S. Bach.
  2. High-Level Planning (The Supernet): This feedforward network operates at the "motif" level (e.g., quarter notes). It analyzes the local melodic context and predicts the class of the next motif to be used. Rather than picking specific notes, it selects the general "vibe" or contour of the sequence.
  3. Low-Level Execution (The Subnet): This network takes the abstract motif class predicted by the Supernet, along with the current harmony, to generate the actual sequence of four sixteenth notes.
  4. Motif Classification: To train these networks, the authors utilized unsupervised learning—specifically Kohonen Self-Organizing Maps—to automatically group musical motifs from the training data into clusters based on their shape and contour.

The Secret Sauce: Musical Representation

A critical factor in MELONET I's success is its data representation. The authors did not simply feed raw MIDI notes into the system. Instead, they used a clever 12-bit interval coding that captured fundamental musical relationships, such as octave invariance and complementary intervals (recognizing, for instance, that an ascending third and a descending sixth are harmonically similar).

Motif Encoding and Harmonic Field

Furthermore, they employed "rotated harmonic fields." This technique aligned the underlying chord structure with the first note of each motif, making it significantly easier for the network to identify which notes were harmonically "safe." By encoding music in a way that reflects actual music theory, the researchers drastically reduced the complexity the neural network had to learn from scratch.

Training and Key Results

The system was trained on a specialized dataset of Baroque music, specifically 12 chorale variations by Johann Pachelbel (totaling 2,220 patterns for the subnet and 555 for the supernet). A validation set of 6 additional Pachelbel variations was used to tune the model.

The "Class" Trade-off: The researchers discovered a natural tension between the Supernet and Subnet. With very few motif classes, the Supernet easily guesses the correct class (high accuracy), but the Subnet struggles to generate precise notes because the classes are too broad. Conversely, with many classes, the Subnet becomes highly precise, but the Supernet’s prediction task becomes much harder. Ultimately, with 5 motif classes, the Supernet achieved ~50% validation accuracy, while with 20 motif classes, the Subnet achieved ~87% validation accuracy.

Stylistic Transfer and Validation: Human experts judged the quality of the generated chorale partitas to be comparable to improvisations by an experienced human organist. To test its generalization, the system was tasked with creating a variation of the non-Baroque melody "Twinkle Twinkle Little Star" (KV 265). It successfully generated a Baroque-style variation, demonstrating that it had captured Pachelbel's stylistic essence rather than merely memorizing training sequences.

Limitations and Future Work

While MELONET I handles motif-level generation exceptionally well, the authors note that it still lacks a true "phrase-level" understanding. Introducing a third, higher-level network could potentially manage the overall structure of an entire musical phrase, pushing global coherence even further. Additionally, the current model focuses on a specific variation type (four sixteenth notes per quarter note), leaving room for future expansion into the styles of other composers like Haydn and Mozart, or even into the complex realm of Jazz improvisation.

Summary updated

BibTeX
@inproceedings{hornel1997melonet,
  author = {Dominik Hörnel},
  title = {{MELONET I: Neural Nets for Inventing Baroque-Style Chorale Variations}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1997},
}
Figure