Abstract
To model the periodicity of beats, state-of-the-art beat tracking systems use "post-processing trackers" (PPTs) that rely on several empirically determined global assumptions for tempo transition, which work well for music with a steady tempo. For expressive classical music, however, these assumptions can be too rigid. With two large datasets of Western classical piano music, namely the Aligned Scores and Performances (ASAP) dataset and a dataset of Chopin's Mazurkas (Maz-5), we report on experiments showing the failure of existing PPTs to cope with local tempo changes, thus calling for new methods. In this paper, we propose a new local periodicity-based PPT, called predominant local pulse-based dynamic programming (PLPDP) tracking, that allows for more flexible tempo transitions. Specifically, the new PPT incorporates a method called "predominant local pulses" (PLP) in combination with a dynamic programming (DP) component to jointly consider the locally detected periodicity and beat activation strength at each time instant. Accordingly, PLPDP accounts for the local periodicity, rather than relying on a global tempo assumption. Compared to existing PPTs, PLPDP particularly enhances the recall values at the cost of a lower precision, resulting in an overall improvement of F1-score for beat tracking in ASAP (from 0.473 to 0.493) and Maz-5 (from 0.595 to 0.838).
Introduction
Beat tracking—the computational process of identifying the pulse of a musical recording—is a foundational task in Music Information Retrieval (MIR). While modern deep learning-based systems have achieved high accuracy on music with relatively stable tempos, such as pop or electronic dance music, they frequently struggle with Western classical music. The primary obstacle is rubato: the expressive, intentional speeding up and slowing down of tempo used by performers to convey emotion and structural nuance.
Standard beat-tracking architectures typically employ a two-stage pipeline. First, a "novelty detection" module (often a neural network) generates a beat activation function. Second, a "post-processing tracker" (PPT) filters these activations to determine the most likely sequence of beats. This research identifies that the bottleneck in classical beat tracking lies not necessarily in the activation functions, but in the rigid global assumptions inherent in traditional PPTs.
The Problem: Global Rigidity vs. Local Expressivity
Traditional PPTs, such as those based on Hidden Markov Models (HMMs) or standard Dynamic Programming (DP), operate under the assumption that tempo changes are rare and occur within a narrow, globally defined range. When faced with the dramatic tempo swings of a Chopin Mazurka, these systems often fail. They may perceive a legitimate expressive slowdown as an error and attempt to "correct" it by sticking to a defunct global tempo, leading to a total loss of synchronization.
Analysis of the Maz-5 dataset (301 recordings of Chopin Mazurkas) reveals that only 13.1% of the tracks possess a "stable" tempo. In contrast, typical evaluation datasets for other genres show stability in over 90% of tracks. This disparity highlights the need for a tracker that prioritizes local context over global constraints.
Proposed Method: PLPDP
The researchers propose Predominant Local Pulse-based Dynamic Programming (PLPDP), a tracker designed to adapt to local tempo variations in real-time. The method consists of three primary components:
- Predominant Local Pulses (PLP): Instead of estimating a single tempo for the entire track, the system calculates local periodicity. By applying sinusoidal kernels to the novelty function within small, overlapping windows, the algorithm identifies the "pulse" dominant in the immediate temporal vicinity.
- Multi-Kernel Combination: To avoid artifacts and ensure robustness across different rhythmic textures, the system utilizes multiple kernel sizes (e.g., 1, 3, and 5 seconds). These are combined through element-wise multiplication, creating a "combined PLP" curve that represents a more reliable estimate of local rhythmic expectations.
- Adaptive Dynamic Programming: In standard DP beat tracking, the penalty for tempo inconsistency is fixed. PLPDP replaces these static parameters with time-varying values derived directly from the PLP curve. This allows the algorithm to adjust its "confidence" and "expectations" dynamically. If the local pulse suggests a tempo shift is occurring, the DP component becomes more flexible, allowing the tracked beat to follow the performer's rubato.
Experimental Results
The PLPDP method was evaluated against state-of-the-art baselines, including the popular madmom HMM tracker, using the ASAP and Maz-5 datasets.
On the Maz-5 dataset, the F1-score jumped from 0.595 (HMM) to 0.838 (PLPDP). This represents a massive leap in performance for highly expressive music. On the larger ASAP dataset, the F1-score improved from 0.473 to 0.493, demonstrating that the local approach is robust across a wider variety of classical styles.
To further isolate the tracker's performance, the researchers conducted a "synthetic test" using an oracle novelty function (perfect beat activations). In this scenario, PLPDP achieved an F1-score of 0.982, while the HMM baseline reached only 0.794. This confirms that the logic of the tracker itself—specifically its inability to handle local tempo shifts—was the primary limiting factor in previous systems.
Discussion and Conclusion
While PLPDP significantly improves recall (the ability to find beats during complex passages), it occasionally does so at the expense of precision, sometimes identifying beats where activations are weak. Furthermore, the system can still experience "metric-level switching," where it inadvertently jumps between tracking the quarter note and the half note.
Despite these hurdles, the success of PLPDP demonstrates that modeling local temporal expectations is key to tracking the "heartbeat" of expressive music. By moving away from rigid global assumptions and toward a flexible, local periodicity-based approach, MIR systems can finally begin to follow the nuanced timing of the world's great classical performers.
Summary updated