Introduction
Generating symbolic music, such as MIDI, that maintains long-term harmonic coherence remains a significant challenge in AI composition. Existing models often suffer from "harmonic ignorance," where individual notes are generated without a clear tonal center, leading to aimless melodies. Furthermore, managing the complexity of multitrack music—where multiple instruments must harmonize simultaneously—requires sophisticated architectural design.
MMT-BERT addresses these issues by introducing a "chord-aware" symbolic representation and a Generative Adversarial Network (GAN) framework. By explicitly teaching the model about underlying harmonies and leveraging a pre-trained music-understanding discriminator, MMT-BERT produces multitrack compositions that are more harmonious and human-like than previous state-of-the-art methods.
Chord-Aware Symbolic Representation
A core innovation of this work is the "quintuple token" representation. Unlike traditional MIDI event sequences, each musical event in MMT-BERT is defined by five distinct features: beat, position/degree, pitch/root, duration/mode, and instrument/extension.
To enforce harmonic structure, the researchers utilize the MusicLang chord analysis model to extract and insert specific Chord Events at the beginning of every bar. These tokens act as a harmonic blueprint, guiding the generator to choose notes that align with the intended tonal center and scale. This "scaffolding" approach ensures that the model does not have to guess the harmony from individual notes.
The MMT-BERT Framework
The MMT-BERT architecture is structured as a GAN, consisting of a generator and a discriminator designed specifically for symbolic music.
Generator: Multitrack Music Transformer (MMT)
The generator is based on the Multitrack Music Transformer (MMT), which utilizes a Transformer-XL backbone. This choice is critical for handling long-range dependencies in music, allowing the model to maintain a "memory" of previous segments and ensure thematic consistency across a piece.
Discriminator: MusicBERT
The discriminator employs a fine-tuned MusicBERT, a model previously pre-trained on massive symbolic music datasets. Unlike standard discriminators trained from scratch, MusicBERT enters the GAN process with a pre-existing "understanding" of musical rules and structures. This allows it to provide more nuanced feedback to the generator, effectively acting as an expert critic.
Training Stability
To stabilize the GAN training process, the authors implement Relativistic Standard Loss. This loss function encourages the generator to produce data that is "more realistic" than the real samples, preventing the discriminator from becoming overwhelmingly powerful too early in the training process and ensuring a steady learning gradient.
Experimental Evaluation
The model was trained on the Symbolic Orchestral Database (SOD), comprising 5,864 pieces. The dataset was partitioned into an 80/10/10 split for training, testing, and validation, with a focus on 4/4 time signatures to facilitate consistent chord extraction.
Quantitative Results
MMT-BERT demonstrated superior performance across several objective metrics compared to baselines like MMM and FIGARO:
- Pitch Class Entropy Similarity (PCES): 99.73%, indicating high tonal stability.
- Scale Consistency Similarity (SCS): 99.64%, confirming that generated notes adhere to the intended scales.
- Groove Consistency Similarity (GCS): 99.66%, reflecting realistic rhythmic patterns.
Qualitative Results
Subjective listening tests involving 18 participants showed that MMT-BERT significantly outperformed the original MMT in categories of Richness and Humanness. While the model showed slight room for improvement in high-level "Structureness," the overall output was judged to be much closer to human-composed music.
Conclusion and Future Work
MMT-BERT successfully demonstrates that explicit harmonic scaffolding and pre-trained discriminators can significantly enhance the quality of AI-generated music. While the model excels at local harmony and multitrack coordination, future work will focus on improving global structures (such as A-B-A song forms) and addressing minor decoding gaps in the final audio conversion. The team also aims to explore multi-modal generation, using text or visual prompts to guide musical style.
Summary updated