TempoExpress: An Expressivity-Preserving Musical Tempo Transformation System

Maarten Grachten, Josep Lluís Arcos, Ramón López de MántarasAAAI 2006

Introduction

In the realm of digital audio, changing the tempo of a recording typically relies on Uniform Time Stretching (UTS). This standard technique scales every note and silence by a constant factor. However, human musicality is rarely linear. As performers change tempo, they adjust phrasing, timing, and rhythm—such as the "swing" in jazz—to maintain expressive intent. This phenomenon challenges the relational invariance hypothesis, which posits that the relative proportions of note durations remain constant across different tempos.

TempoExpress is a Case-Based Reasoning (CBR) system designed to bridge this gap. By moving beyond simple mathematical scaling and learning from a database of real human performances, the system aims to preserve the "soul" of a performance even when the tempo is significantly altered.

Methodology: Case-Based Reasoning for Music

The researchers utilized a CBR paradigm, which solves new problems by retrieving and adapting solutions from similar past cases. The process involves four key stages:

  1. Representation: The system converts a MIDI score and an audio recording into an XML-based melodic description using pitch and onset detection.
  2. Retrieval: Using a modified Levenshtein edit-distance algorithm—optimized to match human perception of musical similarity—the system searches a "case base" of human-performed jazz phrases to find segments similar to the input.
  3. Reuse & Adaptation: Through constructive adaptation, the system breaks retrieved phrases into smaller segments and recombines them to build a "performance annotation." This acts as a set of instructions for deviating from the literal MIDI score to sound expressive at the target tempo.
  4. Synthesis: This annotation generates a new melodic description, which is then synthesized back into audio.

Experimental Results

The authors evaluated TempoExpress (TE) against UTS across 6,364 tempo-transformation problems derived from 14 jazz phrases. The primary metric was the "mean distance to target performance," measuring how closely the transformed audio matched a real human recording at the target tempo.

Figure 1: Mean distance to target performance across various tempo change proportions. The solid line represents TempoExpress, while the dashed line represents Uniform Time Stretching.

As shown in Figure 1, the distance to the target performance increases as the tempo change becomes more extreme, particularly for UTS. The results highlight a significant asymmetry in performance:

  • Slowing Down (Tempo ↓): TempoExpress significantly outperformed UTS. The mean distance to the target human performance was 0.0760 for TE vs. 0.0786 for UTS ($p < .001$). This suggests that the expressive adjustments humans make at slower speeds are complex and better captured by the CBR approach.
  • Speeding Up (Tempo ↑): The results were less decisive. UTS showed a slightly lower mean distance (0.0785) than TE (0.0791), though with lower statistical significance ($p < .05$).
Figure 2: Statistical summary of the Wilcoxon signed-rank test comparing TempoExpress and UTS performance.

Key Insights

The Death of Relational Invariance

If the relational invariance hypothesis were correct, the distance between UTS-transformed audio and a real human performance would remain constant. Instead, the data shows that as tempo deviates from the original, the error for UTS grows. This confirms that music does not scale linearly; expressivity is tempo-dependent.

Data Sparsity and Speed

The system's relative struggle when speeding up music is attributed to "data sparsity." The current case base lacks sufficient examples of high-speed performances. Without a "memory" of how to play fast, the CBR system cannot suggest better alternatives than simple scaling.

Conclusion and Future Work

TempoExpress demonstrates that a data-driven, case-based approach can successfully preserve musical expressivity, particularly when slowing down performances. While currently limited to monophonic saxophone jazz, future work aims to expand the case base to include more diverse tempos and styles, and eventually tackle the complexities of polyphonic instruments like the piano or full ensemble performances.

Summary updated

BibTeX
@inproceedings{grachten2006tempoexpress,
  author = {Maarten Grachten and Josep Lluís Arcos and Ramón López de Mántaras},
  title = {{TempoExpress: An Expressivity-Preserving Musical Tempo Transformation System}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2006},
}
Figure