Rhythm Inference Helping Writing Music Scores

François SchwarzentruberIJCAI 2024

Abstract

This paper presents a novel approach to music notation called "rhythm inference," which automates the determination of note durations from minimalist text-based input. By modeling rhythm as a Constraint Satisfaction Problem (CSP) solved via Mixed Linear Programming (MLP), the proposed abcd language allows composers to focus on melodic content and relative spacing rather than explicit rhythmic values. We describe the mathematical framework, the implementation using Google OR-tools, and the potential for this system to act as a "rhythmic autocorrect" for musical transcription.

Introduction

Traditional music notation software often forces a choice between tedious graphical interfaces and verbose textual formats like Lilypond or ABC. In both cases, the user must explicitly define the duration of every note and rest—a significant cognitive burden during the creative process. This paper introduces "rhythm inference," a technique that bridges this gap by allowing users to write scores using a "markdown-like" language called abcd. The system is designed to "guess" intended rhythms based on partial hints and the relative horizontal spacing of text.

The abcd Language

The abcd language prioritizes human readability and conciseness. Instead of specifying "sixteenth note" or "dotted quarter," a user types note names and uses physical space as a proxy for duration.

Example of the minimalist abcd syntax, showing note names, a partial hint (the dot), and measure boundaries.

As shown in the example above, the syntax is extremely sparse. The system interprets the dot (.) as a hint that a note is dotted, but the exact duration is left to the inference engine to resolve within the constraints of the time signature.

Mathematical Framework

The core of the system is the formalization of rhythm as a Constraint Satisfaction Problem (CSP), solved using Mixed Linear Programming (MLP).

Variables and Domains

Each note $i$ is assigned a variable $\delta_i$ representing its inferred duration. These variables must belong to a domain of standard musical durations (e.g., {1, 1/2, 1/4, 1/8, 1/16}). If a user provides a hint, such as a triplet or a dot, the domain for that specific note is mathematically transformed (e.g., multiplied by 1/3 or 3/2).

Constraints and Optimization

The primary hard constraint is that the sum of all durations within a measure must equal the time signature (e.g., 1.0 for a 4/4 measure). To handle human input, which is often imprecise, the system focuses on relative comparisons. If a user leaves more space for note $A$ than note $B$ in the text editor, the solver attempts to satisfy the constraint $\delta_A \geq \delta_B$. The engine minimizes an error variable to find the "closest" standard musical rhythm that respects these relative rankings.

Implementation

The tool is implemented with a Python backend utilizing Google OR-tools for solving the MLP. The resulting notation is rendered in real-time using the abcjs library.

The rhythm inference tool interface, demonstrating the real-time mapping from spaced text input to a rendered musical score.

The interface allows for a hybrid approach where users can be vague or specific. For instance, adding a number (e.g., a2) can force a specific duration, which the CSP solver then uses as a fixed point while calculating the remaining notes.

Discussion and Results

The rhythm inference engine successfully distinguishes between different rhythmic intents based solely on spacing. For example, wide spacing on an initial note can result in a dotted rhythm, while reducing that space causes the system to automatically re-calculate a consistent score with equal durations.

Key Insights

  • Relative Spacing as Intent: By optimizing for relative order rather than absolute distance, the system becomes robust to "fuzzy" human input.
  • Rhythmic Autocorrect: This framework could significantly improve automatic music transcription by cleaning up the small timing errors inherent in MIDI performances.

Limitations and Future Work

Current limitations include a focus on single-voice notation and a lack of support for anacrusis (pick-up measures). Future work will involve multi-voice constraints to ensure vertical alignment in text corresponds to simultaneous playback, and the implementation of preference models to favor common musical patterns over mathematically valid but unconventional rhythms.

Summary updated

BibTeX
@inproceedings{schwarzentruber2024rhythm,
  author = {François Schwarzentruber},
  title = {{Rhythm Inference Helping Writing Music Scores}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}

Loading related content...

Figure