Local Periodicity-Based Beat Tracking for Expressive Classical Piano Music

Ching-Yu Chiu, Meinard Müller, Matthew E. P. Davies, Alvin Wen-Yu Su, Yi-Hsuan Yang

TL;DR

Standard beat-tracking algorithms often fail on expressive classical music because they rely on rigid, global tempo assumptions. This paper introduces PLPDP, a new post-processing tracker that swaps global constraints for "local periodicity," allowing it to gracefully follow the dramatic tempo swings (rubato) of composers like Chopin, significantly boosting performance on challenging classical datasets.

What Problem Does This Paper Address?

Most beat-tracking systems—the tech that lets a computer "tap its foot" to music—are designed for pop, rock, or electronic music with steady, metronomic beats. These systems typically use a two-stage process: a "novelty detection" module (often a Deep Learning model) that identifies potential beat candidates, and a "post-processing tracker" (PPT) that filters these candidates into a consistent pulse.

The problem? Traditional PPTs (like Hidden Markov Models or standard Dynamic Programming) assume that tempo changes slowly and stays within a narrow global range. In expressive classical music, performers use rubato—intentional, dramatic speeding up and slowing down for emotional effect. To a standard tracker, these expressive shifts look like "errors," causing the system to lose the beat entirely or stick stubbornly to a "global" tempo that no longer exists.

Methods

The authors propose a new tracker called Predominant Local Pulse-based Dynamic Programming (PLPDP). The core innovation is moving from global tempo expectations to local temporal expectations.

  1. Predominant Local Pulses (PLP): Instead of looking at the whole song, the system analyzes the novelty function in small, overlapping windows. It fits sinusoidal kernels to these windows to estimate the local periodicity.
  2. Multi-Kernel Combination: The researchers found that using a single window size (kernel) can create artifacts. They combine multiple kernel sizes (e.g., 1, 3, and 5 seconds) through element-wise multiplication to create a more robust "combined PLP" curve.
  3. Adaptive Dynamic Programming: Traditional Dynamic Programming (DP) beat trackers use a fixed penalty for tempo inconsistency. PLPDP replaces these fixed values with time-varying ones derived from the PLP curve. It essentially tells the algorithm: "Based on the last few seconds, here is the tempo I expect to see next, and here is how confident I am in that expectation."

Datasets

The study evaluates the method on two major datasets of expressive Western classical piano music:

  • Maz-5: 301 recordings of five Chopin Mazurkas. This is a "stress test" for beat trackers due to the extreme rubato inherent in the genre.
  • ASAP (Aligned Scores and Performances): A larger, more diverse dataset containing 502 performances of various Western classical pieces.

The authors highlight that only 13.1% of the Maz-5 tracks have a "stable" tempo (compared to over 90% for typical dance music datasets), highlighting why classical music requires a specialized approach.

Key Insights and Ideas

  • Global is the Enemy of Expressive: The researchers proved that even with a "perfect" (oracle) novelty function, traditional HMM-based trackers can't achieve high scores on Chopin because their internal models are simply too rigid to follow the performer.
  • Mimicking Human Cognition: The PLP approach is inspired by cognitive neuroscience, which suggests the human brain continuously generates "temporal expectations" for upcoming events based on very recent local context.
  • Recall vs. Precision Trade-off: PLPDP significantly improves Recall (finding the beats that are actually there) but sometimes at the cost of Precision (it might occasionally hallucinate a beat where there is only a weak activation). However, in the context of expressive music, the gain in recall is so large that the overall F1-score (the balance of the two) improves dramatically.

Key Results

The PLPDP method outperformed state-of-the-art baselines (like the popular madmom HMM tracker) across the board:

  • Maz-5 Dataset: The F1-score jumped from 0.595 (HMM) to 0.838 (PLPDP). This is a massive leap in a field where improvements are usually measured in single percentage points.
  • ASAP Dataset: The F1-score improved from 0.473 to 0.493. While the gain is smaller than in Maz-5, it demonstrates that the local approach is robust across a wider variety of classical styles.
  • Synthetic Tests: When provided with "perfect" synthetic beat activations, PLPDP achieved an F1-score of 0.982, whereas the standard HMM baseline struggled at 0.794, proving that the bottleneck in classical beat tracking has been the tracker's logic, not just the audio features.

Limitations and Future Work

Despite the success, the authors note a few hurdles:

  • Metric-Level Switching: The system sometimes gets confused between different "levels" of the beat (e.g., suddenly switching from tracking the quarter note to the half note).
  • Front-End Sensitivity: While the tracker is better, it still relies on the initial "novelty function." If the deep learning model at the start of the pipeline fails to detect any note onsets (common in very soft, blurred piano passages), the tracker has nothing to work with.
  • Future Directions: The authors suggest incorporating higher-level musical information, such as pitch, melody, or harmony, to help the tracker stay on the right "metric level" during complex passages.

Summary updated

Abstract

To model the periodicity of beats, state-of-the-art beat tracking systems use “post-processing trackers” (PPTs) that rely on several empirically determined global assumptions for tempo transition, which work well for music with a steady tempo. For expressive classical music, however, these assumptions can be too rigid. With two large datasets of Western classical piano music, namely the Aligned Scores and Performances (ASAP) dataset and a dataset of Chopin's Mazurkas (Maz-5), we report on experiments showing the failure of existing PPTs to cope with local tempo changes, thus calling for new methods. In this paper, we propose a new local periodicity-based PPT, called predominant local pulse-based dynamic programming (PLPDP) tracking, that allows for more flexible tempo transitions. Specifically, the new PPT incorporates a method called “predominant local pulses” (PLP) in combination with a dynamic programming (DP) component to jointly consider the locally detected periodicity and beat activation strength at each time instant. Accordingly, PLPDP accounts for the local periodicity, rather than relying on a global tempo assumption. Compared to existing PPTs, PLPDP particularly enhances the recall values at the cost of a lower precision, resulting in an overall improvement of F1-score for beat tracking in ASAP (from 0.473 to 0.493) and Maz-5 (from 0.595 to 0.838).
BibTeX
@article{chiu2023local,
  author = {Ching-Yu Chiu and Meinard Müller and Matthew E. P. Davies and Alvin Wen-Yu Su and Yi-Hsuan Yang},
  title = {{Local Periodicity-Based Beat Tracking for Expressive Classical Piano Music}},
  journal = {IEEE ACM Trans. Audio Speech Lang. Process.},
  year = {2023},
  url = {https://openreview.net/forum?id=mrZ2V2TiuG},
}

Loading related content...

Figure