Warped Tempo-Ratio Graphs for Octave-Robust Beat Tracking in Expressive Music

Beat trackers rarely fail at random. When they fail, they usually report a pulse at exactly twice, half, or two-thirds the true rate. This is the "tempo octave" error: the true beat is still among the model's top candidates, but the model picks the wrong level of the musical hierarchy. In steady pop and dance music, long-range context fixes this, because the bar repeats at a fixed lag and the model can check which candidate level is consistent across many bars. In expressive classical music, performers stretch and compress time (rubato). The lag to "one bar ago" keeps changing, and the fixed-lag context that resolves octave errors stops lining up. That is one reason trackers that do well on rock and dance drop sharply on Chopin Mazurkas, where only 13.1% of recordings have stable tempo (Local Periodicity-Based Beat Tracking for Expressive Classical Piano Music).

We propose to split the problem in two. The first question is how time is being stretched. It does not require choosing the beat level, because doubling a tempo curve changes nothing about where it speeds up or slows down. The second question is which level is the beat, and it is answered in time that has already been un-stretched. Concretely, we build a graph whose nodes are (time frame, candidate tempo) pairs. Candidate tempi are linked when they stand in small integer ratios (2:1, 3:1, 3:2, 4:1). Time frames are linked at lags measured in estimated beats rather than in seconds. A learned two-dimensional spectral filter on this graph then picks the metrical level that stays coherent along both axes.

Core Claim

The claim is that octave errors in expressive music come largely from a mismatch: rubato moves the musical lags, while existing architectures look at fixed lags. Fixing that mismatch requires temporal edges that stretch with the performance. Adding more model capacity does not. If the claim holds, our model should shrink the gap between octave-strict and octave-tolerant scores much more on rubato-heavy data (Maz-5, ASAP, SMC) than on steady data (Ballroom, GTZAN). It should also do so without the precision loss that local-pulse trackers pay.

The idea draws on unified spatiotemporal graph modeling. That line of work collapses "entities" and "time" into one product graph so that information can flow across both in a single operation (Graph-Time Convolutional Neural Networks: Architecture and Theoretical Analysis). Our "entities" are not instruments or sensors. They are the candidate tempi, and the edges between them encode the integer-ratio structure of meter itself.

Research Program

The input is a log-tempogram. This is a time-by-tempo map of how strongly each candidate pulse rate is present in each frame. We compute it from the frame-level beat activations of a standard neural front end, and we use the same front end for all compared systems. On log-tempo, a 2:1 relation is a fixed shift of one octave and 3:1 a shift of $\log_2 3$. The ratio graph therefore has the same shape at every tempo, which lets parameters be shared across tempi.

Let node $(t,k)$ denote frame $t$ and tempo bin $k$, and let $\phi(t)$ be a "beat-time" coordinate: the cumulative integral of an estimated local tempo curve. The edges are

$$ (t,k)\sim(t,k') \text{ if } \tau_k/\tau_{k'}\in R, \qquad (t,k)\sim(t',k) \text{ if } \phi(t')-\phi(t)\in \{n\,c_k : n=1,\dots,N\}, $$

where $\tau_k$ is the period of bin $k$, $R$ is a small set of musical ratios that includes ternary ones, and $c_k$ is the number of reference beats spanned by one period of bin $k$.

The local tempo curve comes from a first pass, either predominant-local-pulse analysis as in PLPDP or the network's own tempogram argmax. That first pass may well track the wrong octave. This is harmless for the warp: an octave-wrong curve rescales $\phi$ by a constant factor, and the integer-ratio edges still connect the same musical events. This invariance is what lets us use a cheap, octave-ambiguous estimate to build the graph that then resolves the octave.

On this product graph we learn a filter $h(\lambda_r,\lambda_t)$, a response over the ratio-graph and warped-time-graph frequencies. Following the bivariate Bernstein polynomial parameterization of Designing Specialized Two-Dimensional Graph Spectral Filters for Spatial-Temporal Graph Modeling, we evaluate it with polynomial shifts rather than an eigendecomposition. That paper showed on traffic and air-quality data that such filters escape the smoothing ("low-pass") bias of stacked graph convolutions and can form band-pass and comb-like responses. A comb along the ratio axis is exactly the shape that tests whether a candidate's integer multiples and divisors are also present. We use only the method from that work; the graph and the task are new.

The output head predicts, per frame, a distribution over which tempo bin is the beat level, plus a refined beat activation. Beats are decoded with the same dynamic programming used by the baselines, so decoding differences cannot explain the results.

Two prior results constrain the design. First, Graph-Time Convolutional Neural Networks prove that sensitivity to graph perturbations grows with filter order and window length. Our graph is built from an estimated tempo curve, so it is perturbed by construction. We therefore keep the polynomial order low, cap $N$ at a few bars, and measure robustness directly. Second, Spatial-Temporal Fusion Graph Neural Networks for Traffic Flow Forecasting is the closest precedent for elastic temporal edges: it adds Dynamic Time Warping similarity edges offline. Our edges differ in two ways. They are derived from a beat-time coordinate rather than from curve similarity. They are also rebuilt per piece at inference.

Beat Tracking the Graphical Model Way also reasons over a joint (tempo, phase) state, but with hand-set potentials and a one-dimensional chain. It locked onto 2/3 tempo during a 4/4-to-3/4 transition, a failure our ternary ratio edges are meant to address.

Baselines

The naive reference point is the madmom hidden-Markov-model tracker, a standard probabilistic beat decoder. Local Periodicity-Based Beat Tracking reports its F1 at 0.595 on Maz-5 and 0.473 on ASAP. We run it on the same activations as every other system.

The strongest existing method on expressive piano is PLPDP from the same paper. It is dynamic programming whose transition penalties follow multi-kernel local pulse curves, and it reports F1 of 0.838 on Maz-5 and 0.493 on ASAP. It is the main competitor because it already handles rubato well. Its acknowledged weakness is precision lost to spurious insertions, which is exactly where our coherence filter should help. We run PLPDP with its published kernels on our front-end activations. Beating it means higher Maz-5 F1 together with higher precision. Matching its F1 through recall alone would not count.

On the general SMC benchmark, the strongest reported number in our file is 61.3% F1 from BeatFM (BeatFM: Improving Beat Tracking with Pre-trained Music Foundation Model). BeatFM aggregates features from frozen music foundation models. Our goal against it is narrower than outright F1. We aim to match its octave-tolerant score while improving the octave-strict score. We will also test whether adding our graph head on top of BeatFM features helps, which would show the contribution is complementary rather than competing.

For steady and metrically varied music, the reference is Deconstruct, Analyse, Reconstruct, a temporal convolutional network with 2:1 and 4:1 dilations, multi-task beat, downbeat and tempo heads, and Bayesian-network decoding. It reports over 83% strict tempo Accuracy 1 on unseen sets and over 90% CMLt on Ballroom. We retrain it on our training split with the same augmentation and tuning budget.

Beat Transformer uses alternating temporal and instrument attention over demixed stems. We have no isolated reported result on Lacuna for Beat Transformer on SMC, Maz-5 or ASAP, so we run it ourselves.

All learned systems get identical training data, comparable parameter counts, and the same number of hyperparameter trials.

Three ablations isolate the key ingredient. In the first, the time edges sit at fixed lags in seconds instead of beat-time lags. This makes the model essentially a two-dimensional convolution over the log-tempogram. In the second, the ratio set is restricted to powers of two. In the third, the first-pass tempo curve is replaced with one derived from ground-truth beats, which gives an oracle upper bound analogous to PLPDP's oracle-novelty test (0.982 F1).

Why not just…?

Why not just run a 2-D convolutional network over the log-tempogram? This is the most serious threat. On a log-tempo axis, integer ratios are fixed offsets, so convolution kernels can learn harmonic-consistency checks without any graph. Our fixed-lag ablation is exactly this model with matched capacity. If it matches the warped-graph model on Maz-5 and ASAP, the graph framing adds nothing, and we will report it that way. Our hypothesis is that the two models will match on Ballroom and GTZAN but separate on rubato-heavy data, because fixed lags drift off the bar structure as the tempo moves.

Why not just add ternary dilations to the convolutional tracker of Böck and Davies? That addresses compound meter, and we will include it as a baseline variant. It does not address rubato, because dilations are still fixed in seconds. The prediction is that it closes part of our gain on Ballroom waltzes, Carnatic material and the 25-song hard corpus, but not on Maz-5. If it also closes the Maz-5 gap, the warping claim fails.

Why not just post-process a strong network with PLPDP? PLPDP already handles rubato. However, its authors report that its recall gains come with false insertions, and its ASAP gain was small (0.473 to 0.493). That suggests local pulse alone does not settle the metrical level. Because our model emits activations, PLPDP can also decode them. We report that combination, which tests whether the graph fixes level selection while PLPDP handles local phase.

Proposed Experiments

Training uses the standard audio beat corpora: Ballroom, Hainsworth, Beatles, RWC Popular, Harmonix and SMC. We use cross-validation where the cited works do and hold out GTZAN.

The expressive evaluation uses Maz-5 and ASAP audio. Metrical variety is tested on Ballroom's triple-meter dances, the Carnatic set, Candombe and BRID, and the Lang and de Freitas 25-song hard corpus with its meter transitions.

Beyond beat F1, the central diagnostics are two gaps between octave-tolerant and octave-strict scores: the gap between CMLt (strict about the level) and AMLt (tolerant of octave-related levels), and the gap between tempo Accuracy 2 and Accuracy 1. Each gap directly counts how often the right pulse was found at the wrong level. We also report precision and recall separately so the PLPDP comparison is honest.

The decisive check is a two-by-two comparison: warped versus fixed-lag edges, on rubato-heavy versus steady data. The claim survives only if the warped model's reduction of the CMLt–AMLt gap is clearly larger on Maz-5, ASAP and SMC than on Ballroom and GTZAN. It must also beat PLPDP's 0.838 Maz-5 F1 at equal or higher precision.

A second check tests the octave-invariance argument. We deliberately build the graph from first-pass tempo curves that are doubled or halved, and expect accuracy to barely change. We also add increasing noise to the tempo curve and measure the degradation slope. This makes concrete the stability trade-off proved for product-graph convolutions, and it tells us how good the first pass must be. If performance collapses under modest tempo-curve noise, the method depends on an oracle it does not have, and the oracle-gap ablation will show by how much.