Abstract
A competent interpretation of a musical composition presents several non-explicit departures from the written score. Timing variations are perhaps the most important ones: they are fundamental for expressive performance and a key ingredient for conferring a human-like quality to machine-based music renditions. However, the nature of such variations is still an open research question, with diverse theories that indicate a multi-dimensional phenomenon. In the present study, we consider event-shift timing variations and show that sequences of note onset deviations are robust and reliable predictors of the musical piece being played, irrespective of the performer. In fact, our results suggest that only a few consecutive onset deviations are already enough to identify a musical composition with statistically significant accuracy. We consider a mid-size collection of commercial recordings of classical guitar pieces and follow a quantitative approach based on the combination of standard statistical tools and machine learning techniques with the semi-automatic estimation of onset deviations. Besides the reported results, we believe that the considered materials and the methodology followed widen the testing ground for studying musical timing and could open new perspectives in related research fields.
Introduction: The Human Element in Music
In computer music, achieving a "human" sound is a persistent challenge. A perfectly metronomic performance sounds robotic; expressive timing variations—tiny delays and anticipations known as note onset deviations—are what breathe life into music. The central question is whether these deviations are purely idiosyncratic to the performer or if they are "generative," dictated by the underlying musical structure. This research explores the hypothesis that timing deviations serve as unique "signatures" of the musical piece itself, persisting across different interpretations.
Methodology
The study utilized a dataset of 100 professional recordings of 10 classical guitar pieces, featuring 82 unique performers. The researchers developed a workflow to isolate timing patterns from performance quirks.
The process involved several sophisticated steps:
- Semi-automatic Onset Detection: Extracting precise onset times from audio and comparing them to the mechanical timing of the score using the Aubio library.
- Normalization: Deviations were normalized to zero mean and unit variance to focus on timing patterns rather than absolute tempo.
- Classification: Seven machine learning models (including SVM, Naive Bayes, and K-Nearest Neighbors) were tested to identify pieces based solely on sequences of these timing shifts.
Key Insights: Timing as a Fingerprint
The analysis revealed several critical properties of onset deviations that distinguish them from random noise:
- Non-Gaussian Nature: Deviations do not follow a standard bell curve, suggesting they are structured expressive choices rather than random motor noise.
- Context Dependency: A timing shift cannot be predicted by a single note's pitch or duration; it depends on the sequence's context.
- Temporal Order: Shuffling the sequence of deviations causes classification accuracy to plummet, proving that the specific "rhythm" of deviations is the identifying signature.
Results and Performance
The study successfully demonstrated that timing deviations can act as a musical fingerprint. Even with a sequence of only 5 notes, all classifiers significantly outperformed a random baseline.
As shown in the figure above, accuracy scales with the length of the deviation sequence ($l$). For longer sequences (approx. 170 notes), top-performing models like Naive Bayes and SVM reached roughly 80% accuracy. Remarkably, these timing deviations were found to be as effective as—or even superior to—actual written note durations for piece identification.
Conclusion and Future Directions
This research proves that the way humans stretch and compress time is characteristic of the musical piece itself. While currently focused on classical guitar, future work could expand this to larger datasets and different instruments (e.g., bowed strings). These findings have significant implications for "cover song" identification and the development of more realistic MIDI-to-audio synthesis by applying learned structural timing signatures.
Summary updated