Acquisition of Human Feelings in Music Arrangement

IJCAI 1997

TL;DR

This paper tackles the challenge of automatically acquiring and replicating human feelings in music arrangement. By developing a system that learns to predict and modify musical arrangements based on user-defined emotional impressions, the authors demonstrate a novel approach to creating more personalized and emotionally resonant music.

What Problem Does This Paper Address?

Expressing and replicating human feelings, or "Kansei," is notoriously difficult. While traditional psychological methods can analyze feelings associated with existing objects, they struggle to predict feelings for new creations or to generate objects that evoke specific emotions. This is particularly true in creative fields like music arrangement, where subjective impressions are paramount. The authors aim to bridge this gap by developing a system that can learn these implicit emotional connections and use them to automatically arrange music, moving beyond simple mimicry to a more nuanced understanding of emotional impact.

Methods

The paper proposes a two-pronged approach:

  1. Mimicking Arrangements (Section 3): The system first learns to mimic human arrangements. It analyzes existing scores, identifying chord progressions and their associated musical functions (tonic, dominant, etc.). This information is used to train a "chord analyzer" and an "automatic arranger." A "theory refinement system" then enhances these arrangements by introducing decorations and improving techniques, aiming to transfer the impressions of human arrangements to new ones.
  2. Music Recognition Model (Section 4): To move beyond mimicry and capture human feelings, the authors develop a music recognition model. This involves:
  3. Data Collection: Subjects evaluate musical pieces using a semantic differential (SD) method with adjective pairs like "bright-dark," "clear-unclear," etc., on a 7-grade scale.
  4. Inductive Logic Programming (ILP): An ILP system is used to derive models that predict these evaluations. The system analyzes musical features (melody and chords) and their relationships.
  5. Background Knowledge: The ILP system leverages background knowledge about musical elements (pitch, duration, chord roots, functions) to establish rules.
  6. Prediction and Modification (Section 5): Once a user's emotional profile is learned, the system can predict their evaluation of new pieces and modify existing arrangements to achieve a specified impression. Modifications prioritize preserving the original score's essence while adjusting elements that contribute to the desired (or opposite) impression.

Datasets

The paper describes the use of:

  • Human-arranged music scores: Used for training the mimicking arrangement system.
  • 100 well-known music pieces (8 bars each, without modulation): Used for training and evaluating the music recognition model. Subjects evaluated 85 of these pieces, and the system predicted evaluations for the remaining 15.
  • 94 well-known music pieces (4 or 8 bars each, without modulation): Used for experiments in arrangement modification.

No new benchmarks were explicitly introduced, but the collected subject evaluations on these music pieces serve as a dataset for training and testing the emotional prediction and modification capabilities.

Key Insights and Ideas

  • Bridging the Objective-Subjective Gap: The core idea is to quantify subjective human feelings by learning the relationship between objective musical features and emotional responses.
  • ILP for Musical Structure: Inductive Logic Programming proves to be a powerful tool for capturing the complex relational structures within music, outperforming neural networks or decision trees for this task.
  • Hierarchical Approach: The system progresses from mimicking existing arrangements to actively predicting and modifying music based on learned emotional profiles, demonstrating a sophisticated understanding of the problem.
  • Impression Modification is Easier in Opposite Direction: The paper suggests that it's easier to modify a piece to evoke the opposite of its current impression (e.g., from bright to less bright) than to enhance an existing impression significantly.

Key Results

  • The music recognition system achieved an average prediction accuracy of 70% for user evaluations across various adjective pairs.
  • Differences between predicted and actual evaluations were generally between 1.0 and 2.0 on a 7-grade scale.
  • The system was particularly good at predicting impressions related to "bright-dark" based on chords (specifically major/minor thirds), while "fast-slow" impressions were more strongly linked to melody and note duration.
  • In arrangement modification experiments, intended impressions were produced on average 60% of the time.
  • Impressions were improved in 60% of arrangements, with a notable tendency for improvement in "bright-dark" criteria.
  • 80% of arrangements could be successfully modified to cause the opposite impression.

Limitations and Future Work

  • Subjectivity Variance: The paper acknowledges that impressions can vary significantly among subjects, especially for criteria like "fast-slow" and "stable-unstable." While the system learns these subject-dependent variations well, further research into discerning these nuances more deeply is suggested.
  • Melody-based Impressions: The system's current strength lies in chord-based impression prediction. The authors are actively working on analyzing melody sequences to better capture impressions like "fast-slow."
  • Practical Application: The authors envision this system being employed in KARAOKE systems for generating user models, selecting appropriate music, or even suggesting background visuals.
  • Broader Musical Theories: Incorporating more established theories from musicology and psychology could further enhance the system's capabilities for composing and analyzing music.

Summary updated

BibTeX
@inproceedings{1997acquisition,
  title = {{Acquisition of Human Feelings in Music Arrangement}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1997},
}
Figure