Pianist Identification Using Convolutional Neural Networks

Jingjing Tang, Geraint A. Wiggins, György Fazekas

Abstract

This paper presents a comprehensive study of automatic performer identification in expressive piano performances using convolutional neural networks (CNNs) and expressive features. Our work addresses the challenging multi-class classification task of identifying virtuoso pianists, which has substantial implications for building dynamic musical instruments with intelligence and smart musical systems. Incorporating recent advancements, we leveraged large-scale expressive piano performance datasets and deep learning techniques. We refined the scores by expanding repetitions and ornaments for more accurate feature extraction. We demonstrated the capability of one-dimensional CNNs for identifying pianists based on expressive features and analyzed the impact of the input sequence lengths and different features. The proposed model outperforms the baseline, achieving 85.3% accuracy in a 6-way identification task. Our refined dataset proved more apt for training a robust pianist identifier, making a substantial contribution to the field of automatic performer identification.

Introduction

Every master pianist possesses a unique "musical fingerprint"—a distinctive combination of phrasing, timing, and dynamics that distinguishes their interpretation from others. Identifying these nuances computationally is a core challenge in Music Information Retrieval (MIR). Previous methods often relied on small datasets or computationally heavy Recurrent Neural Networks (RNNs) that were slow to train and prone to noise from unrefined score alignments. This study introduces a high-performance 1D Convolutional Neural Network (CNN) that achieves state-of-the-art accuracy by analyzing refined expressive performance features.

Methods

The proposed system utilizes a one-dimensional CNN architecture optimized for sequential musical data. Unlike 2D CNNs used in image processing, 1D CNNs excel at capturing local temporal patterns in piano performances.

Score Refinement and Feature Extraction

To ensure precise alignment between the "ideal" score and the "expressive" performance, the researchers manually expanded repetitions and ornaments (e.g., trills) in the MIDI files. They extracted 13 features per note, including note-wise features (pitch, velocity, timing) and deviation features (differences in timing and volume relative to the literal score).

Model Architecture

The model consists of five convolutional layers followed by a dense layer, utilizing ReLU activation, batch normalization, and dropout to mitigate overfitting. The architecture contains approximately 6.1 million trainable parameters and offers significant improvements in training efficiency compared to hierarchical RNN models.

The 1D CNN architecture used for pianist identification, showcasing convolutional, pooling, and dense layers.

Datasets

The study employed the ATEPP (Automatic Transcription of Expressive Piano Performance) dataset, focusing on six legendary pianists: Sviatoslav Richter, Alfred Brendel, Friedrich Gulda, Claudio Arrau, Daniel Barenboim, and Wilhelm Kempff. Two subsets were curated:

  • ID-400: A cleaned version of a standard literature subset (457 performances).
  • ID-1000: A larger, more robust subset (1,001 performances) used to test scalability.

Results and Discussion

The model achieved a state-of-the-art accuracy of 85.3% on the ID-400 dataset, surpassing previous RNN-based baselines. A key finding was that the model performed significantly better when analyzing full musical pieces rather than short segments (e.g., 400 or 1,000 notes), suggesting that a pianist's stylistic signature is distributed across the entire performance structure.

Data Robustness

By moving from the ID-400 to the ID-1000 dataset, the researchers demonstrated that increased data volume leads to higher stability. The larger dataset significantly reduced the model's sensitivity to how data was split, resulting in more reliable performance metrics across different experimental trials.

Comparison of identification accuracy across five different data splits for the ID-1000 and ID-400 datasets.

Conclusion

This work demonstrates the efficacy of 1D CNNs for automatic performer identification. While currently limited to six pianists and specific classical composers, the results provide a strong foundation for future smart musical systems and the Internet of Musical Things (IoMusT). Future research will explore the model's ability to generalize across different composers and larger performer pools to determine if the system is capturing universal performer style.

Summary updated

BibTeX
@inproceedings{tang2023pianist,
  author = {Jingjing Tang and Geraint A. Wiggins and György Fazekas},
  title = {{Pianist Identification Using Convolutional Neural Networks}},
  booktitle = {Is2},
  year = {2023},
  doi = {10.1109/ieeeconf59510.2023.10335427},
}
Figure