Score difficulty analysis for piano performance education based on fingering

Pedro Ramoneda, Nazif Can Tamer, Vsevolod Eremenko, Xavier Serra, Marius Miron

TL;DR

This paper introduces a novel dataset and methodology for classifying piano score difficulty based on fingering techniques. By leveraging piano fingering features and interpretable machine learning models, the authors achieve accurate difficulty classification and provide valuable local feedback for music education.

What Problem Does This Paper Address?

Classifying music by performance difficulty is an emerging area within Music Information Retrieval (MIR), with significant potential for music education. While existing approaches often rely on instrument-agnostic features, this paper argues for the importance of piano technique, specifically fingering, as a key predictor of difficulty. The challenge lies in developing methods that can accurately assess this technical difficulty, provide granular feedback, and be interpretable for educational purposes. Furthermore, a lack of publicly available, well-annotated datasets for this task has hindered research.

Methods

The authors propose a methodology centered around piano technique features derived from two distinct fingering approaches:

  • Pianoplayer [20]: A knowledge-driven system using local search and dynamic programming to generate fingerings based on score analysis and hard-coded constraints. This yields features related to finger assignment (P-F) and finger velocity (P-v).
  • Nakamura et al. [12]: A data-driven approach using a Hidden Markov Model (HMM) trained on expert-annotated data. This provides features for finger assignment (N-F) and transition probabilities between fingerings (N-P).

As a baseline, they also include a simple note onset representation (K) that is agnostic to piano technique.

These features are then fed into two interpretable machine learning classifiers:

  • XGBoost (Gradient-Boosted Trees): Applied to short-term score segments (windows) for classification. Predictions are then averaged across the entire piece to derive a global difficulty score. This method offers interpretability through decision trees.
  • DeepGRU (GRU Neural Network with Attention): A recurrent neural network designed to handle time dependencies and variable-length sequences. The attention mechanism allows the model to focus on specific notes that are most important for difficulty assessment, providing a form of interpretable feedback.

Datasets

The paper introduces the Mikrokosmos-difficulty dataset, a novel benchmark for piano score difficulty analysis. It comprises 146 piano pieces from Béla Bartók's Mikrokosmos collection, specifically excluding four-hand pieces. The dataset is unique in that all pieces are by a single composer, reducing style-based biases and focusing on technical difficulty. It offers three levels of difficulty labels:

  1. Composer's order: The original ranking by Béla Bartók.
  2. Publisher's book divisions: Groupings based on the original publication volumes.
  3. Henle difficulty labels: A 1-9 scale from the publisher Henle, tailored for professionals.

The dataset is available for download.

Key Insights and Ideas

  • Fingering is a strong predictor of piano difficulty: The study demonstrates that features derived from piano fingering algorithms significantly outperform a baseline that only considers note information. This highlights the crucial role of technical execution in perceived difficulty.
  • Interpretable models are key for educational applications: The use of XGBoost and DeepGRU with attention allows not only for accurate classification but also for generating localized feedback. This is invaluable for music students and teachers to identify and address specific challenging passages.
  • Local difficulty feedback is more informative than global scores: The paper emphasizes the benefit of providing feedback on a segment-by-segment basis, enabling targeted practice and improvement.
  • Pianoplayer features show promise: Features derived from the Pianoplayer system (P-F, P-v) generally perform better than those from Nakamura et al. (N-F, N-P), suggesting the effectiveness of its underlying fingering generation logic.

Key Results

  • Classification Accuracy: The DeepGRU model with attention achieved the highest 3-class classification accuracy, reaching up to 68±10% on the test set using the P-v feature. XGBoost (averaged) also performed well, with P-v yielding 62±6% accuracy.
  • Ranking Correlation: Both XGBoost (averaged) and DeepGRU models showed strong Spearman rank correlation coefficients with the Bartók and Henle difficulty rankings, indicating their ability to predict overall score difficulty. For instance, DeepGRU with P-v achieved a correlation of 0.86±0.05 with the Henle rankings.
  • Feature Importance: Velocity (P-v) and transition probabilities (N-P) were found to be more informative features than finger assignments (P-F, N-F) alone.
  • Window Size Insignificance: For XGBoost, the ablation study on window size revealed that variations in window size did not significantly impact performance, leading the authors to adopt the Pianoplayer's context window of 9 notes.

Limitations and Future Work

  • Dataset Size and Variance: The authors acknowledge that the relatively small size of the Mikrokosmos-difficulty dataset can lead to overfitting and large standard deviations in experimental results.
  • Future Work: The authors plan to extend this research by incorporating larger score datasets with difficulty annotations and exploring difficulty analysis for other musical instruments. They also aim to further develop the online demo for visualizing segmental difficulty.

Summary updated

Abstract

In this paper, we introduce score difficulty classification as a sub-task of music information retrieval (MIR), which may be used in music education technologies, for personalised curriculum generation, and score retrieval. We introduce a novel dataset for our task, Mikrokosmos-difficulty, containing 147 piano pieces in symbolic representation and the corresponding difficulty labels derived by its composer Béla Bartók and the publishers. As part of our methodology, we propose piano technique feature representations based on different piano fingering algorithms. We use these features as input for two classifiers: a Gated Recurrent Unit neural network (GRU) with attention mechanism and gradient-boosted trees trained on score segments. We show that for our dataset fingering based features perform better than a simple baseline considering solely the notes in the score. Furthermore, the GRU with attention mechanism classifier surpasses the gradient-boosted trees. Our proposed models are interpretable and are capable of generating difficulty feedback both locally, on short term segments, and globally, for whole pieces. Code, datasets, models, and an online demo are made available for reproducibility
BibTeX
@inproceedings{ramoneda2022score,
  author = {Pedro Ramoneda and Nazif Can Tamer and Vsevolod Eremenko and Xavier Serra and Marius Miron},
  title = {{Score difficulty analysis for piano performance education based on fingering}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2022},
  doi = {10.1109/icassp43922.2022.9747223},
}
Figure