Responding to the Call: Exploring Automatic Music Composition Using a Knowledge-Enhanced Model

Zhejing Hu, Yan Liu, Gong Chen, Xiao Ma, Shenghua Zhong, Qianwen LuoAAAI 2024 · DOI

TL;DR

This paper introduces a novel approach to automatic music composition that focuses on the "call-and-response" musical technique, aiming to enhance creativity beyond mere note generation. They present a new dataset of call-response pairs and a knowledge-enhanced model (CRG) that leverages musical knowledge to generate more engaging and artistic responses.

What Problem Does This Paper Address?

Automatic music composition has made significant strides, but current models often struggle with complex compositional techniques that imbue music with emotional depth and creativity. One such technique, call-and-response, is fundamental to human musical expression, creating a dialogue between musical phrases. However, this technique is largely underrepresented in machine-generated music. The authors highlight that while existing models can produce technically correct music, they often lack the nuanced creativity found in human compositions, leading to music that is merely "correct" rather than "artistic." This gap is exacerbated by the limited availability of specialized datasets for training models on such intricate musical structures.

Methods

The core of this work is the Call-Response Generator (CRG), a model designed to produce musical responses to a given "call." CRG is built upon a sequence-to-sequence architecture, comprising:

  • Compositional Encoder-Decoder: This standard component maps the input call (a variable-length music sequence) into a fixed-length vector representation and then decodes this representation into a variable-length response.
  • Knowledge-Enhanced Compositional Module: This is the novel aspect. It leverages a "music knowledge" base (a collection of music pieces) to guide the generation process. The process involves:
  • Knowledge Selection: Identifying candidate music phrases from the knowledge base that exhibit rhythmic, melodic, and harmonic similarities to the input call. This selection is guided by a "selection score" derived from these similarities.
  • Knowledge Incorporation: Integrating the representations of these selected candidates into the encoder's output before it's passed to the decoder. This allows the model to draw inspiration from existing musical patterns that align with the call.

The model is trained using two objectives: a knowledge-enhanced objective (predicting the ground-truth response) and an autoencoder-based objective (predicting music candidates), combined using a hyperparameter.

Datasets

  • Call-Response Dataset (CRD): The authors introduce a new dataset containing 19,155 annotated call-response pairs, totaling over 108 hours of music. This dataset is derived from the Pop909 dataset and is specifically curated to include these musical dialogues.
  • Pop1K7 Dataset: Used as the "music knowledge" base for the knowledge-enhanced module.

Key Insights and Ideas

  • Call-and-response as a creativity enhancer: The paper posits that focusing on specific compositional techniques like call-and-response is crucial for moving beyond technically correct music to more artistic and engaging machine compositions.
  • Leveraging external musical knowledge: The CRG's success hinges on its ability to draw inspiration from a curated knowledge base, mirroring how human musicians learn from existing music. This is a departure from models that solely rely on reconstructing input data.
  • Similarity-based knowledge selection is key: The experiments suggest that using "similar knowledge" (candidates that resemble the input call) yields the most significant improvements, reinforcing the idea that musical coherence and connection are vital for effective call-and-response.
  • Trade-off between diversity and quality: Increasing the number of knowledge candidates can lead to more diverse responses but may also introduce a trade-off with overall music quality.

Key Results

  • Objective Metrics: The CRG model significantly outperforms existing models (MT, CPT, HAT) on objective metrics like Rhythm Quality (RQ), Melody Quality (MQ), and Harmony Quality (HQ), achieving scores of 0.59, 0.57, and 0.72 respectively, with statistically significant improvements.
  • Subjective Evaluation: Human listeners rated the CRG-generated music highly across quality, groove, coherence, and creativity. Notably, while human-composed music received slightly higher ratings for quality, groove, and coherence, CRG was perceived as significantly more creative.
  • Response Variety: The CRG can generate a diverse range of response effects (growth, extension, liquidation, conversion, condensation) to a single call, demonstrating its ability to create varied musical dialogues, as illustrated in Figure 7.

Limitations and Future Work

  • Human vs. Machine Creativity: The authors acknowledge that their model does not necessarily generate music with greater overall creativity than human composers but highlights its ability to introduce diverse and creative responses within a single piece.
  • Bias in Evaluation: The paper mentions the need to explore bias in evaluation methods for computational creativity.
  • Future Directions: The authors plan to enhance their approach by including control over specific response effects and further exploring bias in evaluation. The code and dataset are publicly available.

Summary updated

BibTeX
@inproceedings{hu2024responding,
  author = {Zhejing Hu and Yan Liu and Gong Chen and Xiao Ma and Shenghua Zhong and Qianwen Luo},
  title = {{Responding to the Call: Exploring Automatic Music Composition Using a Knowledge-Enhanced Model}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i1.27807},
}
Figure