XiaoIce Band: A Melody and Arrangement Generation Framework for Pop Music

Hongyuan Zhu, Qi Liu, Nicholas Jing Yuan, Chuan Qin, Jiawei Li, Kun Zhang, Guang Zhou, Furu Wei, Yuanchun Xu, Enhong Chen

Abstract

With the development of knowledge of music composition and the recent increase in demand, an increasing number of companies and research institutes have begun to study the automatic generation of music. However, previous models have limitations when applying to song generation, which requires both the melody and arrangement. Besides, many critical factors related to the quality of a song such as chord progression and rhythm patterns are not well addressed. In particular, the problem of how to ensure the harmony of multi-track music is still underexplored. To this end, we present a focused study on pop music generation, in which we take both chord and rhythm influence of melody generation and the harmony of music arrangement into consideration. We propose an end-to-end melody and arrangement generation framework, called XiaoIce Band, which generates a melody track with several accompany tracks played by several types of instruments. Specifically, we devise a Chord based Rhythm and Melody Cross-Generation Model (CRMCG) to generate melody with chord progressions. Then, we propose a Multi-Instrument Co-Arrangement Model (MICA) using multi-task learning for multi-track music arrangement. Finally, we conduct extensive experiments on a real-world dataset, where the results demonstrate the effectiveness of XiaoIce Band.
BibTeX
@inproceedings{zhu2018xiaoice,
  author = {Hongyuan Zhu and Qi Liu and Nicholas Jing Yuan and Chuan Qin and Jiawei Li and Kun Zhang and Guang Zhou and Furu Wei and Yuanchun Xu and Enhong Chen},
  title = {{XiaoIce Band: A Melody and Arrangement Generation Framework for Pop Music}},
  booktitle = {ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  year = {2018},
  doi = {10.1145/3219819.3220105},
}

Loading related content...

Figure