Introduction
HARMONET is a hybrid AI system designed to harmonize melodies in the signature style of J.S. Bach. By combining neural networks for stylistic "creativity" with symbolic algorithms for musical "bookkeeping," it produces four-part chorales that professional musicians have judged to be on par with human improvisations.
The "Aesthetic Gap" in Computer-Generated Music
For decades, researchers used expert systems—complex sets of "if-then" rules—to compose music. While these systems could strictly follow the formal rules of music theory (such as avoiding parallel fifths), the resulting music often felt robotic or "aesthetically offensive."
The core challenge is that musical style is notoriously difficult to describe purely through symbols and rigid rules. There is an elusive quality to how a master like Bach chooses harmonies that goes beyond simple "dos and don'ts." HARMONET seeks to bridge this gap by using connectionist models (neural networks) to learn the "soul" of the style directly from examples, rather than relying solely on a manual of constraints.
Methods: A Hybrid Architecture
The researchers implemented a hybrid architecture that decomposes the complex task of composition into manageable sub-functions. This approach mirrors how human musicians often think: deciding on the "vibe" or chord progression before worrying about every individual note.
- Task Decomposition: Instead of generating a full score at once, the system works in distinct layers:
- Harmonic Skeleton: A neural network predicts the underlying chord progression (the "thoroughbass") for each quarter note.
- Chord Skeleton: A symbolic "generate and test" algorithm fills in the middle voices (alto and tenor) based on the predicted harmony, ensuring no technical rules are broken.
- Ornamentation: A final neural network adds eighth-note flourishes (quavers) to make the music feel more fluid and "Bach-like."
- Novel Pitch Coding: Standard numerical representations of pitch often fail to capture musical relationships. The authors developed a coding scheme where a note is represented by the set of harmonic functions it belongs to (e.g., Tonic, Dominant, Subdominant). This allows the network to "understand" the functional role of a note within a key.
- Sliding Window Architecture: The system uses backpropagation networks with a sliding window approach. To decide the harmony at time t, the network looks at the melody notes at t-1, t, and t+1, as well as the three previously chosen harmonies to provide external feedback.
Training and Datasets
The system was trained on the gold standard of Western harmony: the 389 Chorales of J.S. Bach. The authors focused on two specific training sets: 20 chorales in major keys and 20 in minor keys. By using a sliding window across these chorales, they generated approximately 1,000 training examples per set. The system was then evaluated on melodies not present in the training set—including "Happy Birthday"—to test its generalization capabilities.
Key Insights and Results
- Professional Validation: An audience of music professionals judged HARMONET’s output to be at the level of an improvising church organist, a significant achievement for computer-generated music.
- Hybridity is Essential: The paper argues that neural nets are excellent at "creative" pattern recognition, but symbolic algorithms are better at "bookkeeping" (e.g., ensuring a bass singer doesn't have to hit a note outside their physical range). Combining them yields better results than either alone.
- Efficiency and Performance: The networks achieved reasonable convergence in just 50 to 100 epochs of training. Furthermore, HARMONET compared favorably to "CHORAL," a famous and much more complex rule-based expert system of the era, despite requiring significantly less hard-coded musical expertise to build.
Limitations and Future Work
While groundbreaking, HARMONET has limitations. Because the system relies on a local sliding window, it can occasionally lose the "big picture" or global consistency of a long musical piece, though external feedback loops help mitigate this. Additionally, HARMONET is a specialist tuned specifically to the four-part chorale style of the Baroque era. Finally, the system still relies on a "symbolic crutch" to prevent illegal musical moves. Developing a purely connectionist approach that learns both style and formal constraints simultaneously remains an exciting challenge for future research.
Summary updated