Music Intelligence

Xiaobing Li, Xiaohong Guan, Yun Tie, Xinran Zhang, Qingwen ZhouCommunications 2024 · DOI · OpenAlex

This document represents the proceedings of the Second Summit on Music Intelligence (SOMI 2023), held in Beijing. Rather than a single study, it is a curated collection of research at the intersection of Artificial Intelligence, neuroscience, and musicology.

TL;DR

SOMI 2023 showcases the evolution of "Music Intelligence" from simple pattern mimicry to a deeply interdisciplinary field. The proceedings highlight breakthroughs in AI-driven music therapy, multimodal emotion recognition in conducting, and new hierarchical models for structured symbolic music generation.

What Problem Does This Paper Address?

While AI has made massive strides in audio generation, the field of "Music Intelligence" faces a significant "semantic gap." Computers can generate sounds that resemble music, but they often struggle with:

  1. Structural Logic: Maintaining long-term coherence and hierarchical structure in compositions.
  2. Therapeutic Efficacy: Moving beyond "background music" to create sounds that can measurably improve clinical outcomes (e.g., attention or mood).
  3. Multimodal Understanding: Integrating the physical gestures of a conductor or the nuances of traditional notation (like the Chinese Guqin) into digital workflows.

The researchers in this volume aim to bridge these gaps by combining rigorous engineering with insights from brain science and music theory.

Methods

The proceedings are divided into three core methodological tracks:

  • Neural Science and Therapy: Researchers utilize fMRI and EEG to study how musical training changes brain connectivity. On the application side, they employ the "ISO principle" (matching music to a patient's current physiological/emotional state) and binaural beats to develop AI systems for clinical therapy.
  • Multimodality and Optimization: To improve how AI "sees" music, one study introduces Quantum Particle Swarm Optimization (QPSO) to fuse data from facial expressions and physical gestures during ensemble conducting.
  • Generative Modeling: The volume moves beyond basic Transformers toward more structured approaches. Key techniques include Hierarchical Relative Positional Encoding (HRPE) for symbolic music and CoCoFormer, a controllable, feature-rich model for polyphonic generation.

Datasets

The volume introduces and utilizes several specialized datasets:

  • Chinese Chorales Dataset: A newly introduced, high-quality benchmark specifically designed for music score generation.
  • Guqin Jianzi Notation: A dataset focused on the transcription of traditional Chinese instrument notation using language models.
  • Novel AI Datasets: The preface notes a specific call for "novel music AI datasets" to address the lack of diversity in existing Western-centric training data.

Key Insights and Ideas

  • Music as a Clinical Tool: AI-generated music is being tailored for "Music Healing." By using the ISO principle, AI can dynamically adjust its output to guide a patient from a state of distress to a state of calm.
  • The "Structural" Shift: There is a clear move away from "flat" sequence generation. By using hierarchical encoding (HRPE), researchers are finding that AI can better understand the relationship between a single note, a phrase, and an entire movement.
  • Cross-Cultural Digitization: The work on Guqin notation shows that "Music Intelligence" is being used to preserve and digitize complex, non-Western musical traditions that do not fit into standard MIDI or staff notation.

Key Results

  • Selectivity: The summit was highly competitive, with a 37.0% acceptance rate (10 papers accepted out of 27 valid submissions in the science and technology track).
  • Composition Systems: The "C2-MAGIC" system demonstrated success in generating multi-track accompaniments that are both "chord-controllable" and interpretable, moving away from "black box" generation.
  • Clinical Impact: Preliminary findings in the therapy track suggest that AI-generated binaural beats can measurably improve children's attention spans within the context of music therapy.

Limitations and Future Work

  • Integration of Art and Science: The editors note that 15 submissions in the "art and social science" track were reviewed separately and not included in this technical manuscript. A future challenge is more deeply integrating these qualitative insights into the quantitative AI models.
  • Industry Standards: While the papers present "prototypes," the preface acknowledges a need for "industry standards" for music AI to ensure these tools can move from the lab to professional studios and hospitals.
  • Generalization: Many of the generative models are currently optimized for specific genres (like popular songs or chorales); scaling these to handle the full complexity of orchestral or avant-garde music remains an open goal.

Summary updated

BibTeX
@inproceedings{li2024music,
  author = {Xiaobing Li and Xiaohong Guan and Yun Tie and Xinran Zhang and Qingwen Zhou},
  title = {{Music Intelligence}},
  booktitle = {Communications},
  year = {2024},
  doi = {10.1007/978-981-97-0576-4},
}

Loading related content...

Figure