HumTrans: A Novel Open-Source Dataset for Humming Melody Transcription and Beyond

Shansong Liu, Xu Li, Dian Li, Ying Shan

Abstract

This paper introduces the HumTrans dataset, which is publicly available and primarily designed for humming melody transcription. The dataset can also serve as a foundation for downstream tasks such as humming melody based music generation. It consists of 500 musical compositions of different genres and languages, with each composition divided into multiple segments. In total, the dataset comprises 1000 music segments. To collect this humming dataset, we employed 10 college students, all of whom are either music majors or proficient in playing at least one musical instrument. Each of them hummed every segment twice using the web recording interface provided by our designed website. The humming recordings were sampled at a frequency of 44,100 Hz. During the humming session, the main interface provides a musical score for students to reference, with the melody audio playing simultaneously to aid in capturing both melody and rhythm. The dataset encompasses approximately 56.22 hours of audio, making it the largest known humming dataset to date. The dataset will be released on Hugging Face, and we will provide a GitHub repository containing baseline results and evaluation codes.

Introduction

Automatic Music Transcription (AMT)—the process of converting audio into musical notation—has seen significant performance leaps for instruments like the piano. However, vocal transcription, specifically humming, remains a "boss level" challenge for AI. Humming is inherently difficult because human voices lack the pitch stability and sharp "onsets" (the start of a note) found in percussive or keyed instruments.

Furthermore, high-quality data for this task has historically been scarce. Previous datasets are often under 10 hours long or suffer from broken accessibility. Without a large, diverse, and well-labeled dataset, developing robust humming-to-sheet-music applications has been nearly impossible. To bridge this gap, we present HumTrans, the largest open-source dataset for humming melody transcription to date.

Data Collection Methodology

To ensure high-quality data, we developed a custom web-based pipeline for systematic collection. We curated 500 musical compositions across various genres, ensuring the melodies were "hummable" (avoiding extreme pitch jumps). We recruited 10 music majors or proficient instrumentalists to ensure pitch accuracy and rhythmic timing.

Web-based recording interface used for data collection.

As shown in the interface above, participants hummed using the syllable "Da-Da-Da" while listening to a reference track and viewing a musical score. This specific syllable helps models identify note onsets more clearly than a continuous "Mmm." The rhythmic consistency of the "D" consonant provides a natural anchor for note onsets without requiring manual frame-by-frame annotation. Recordings were sampled at 44.1 kHz and filtered using DNSMOS (a deep noise suppression mean opinion score) to ensure high audio quality.

The HumTrans Dataset

HumTrans represents a significant scaling up of available resources for humming transcription. It contains 56.22 hours of audio (14,614 recording files) derived from 1,000 music segments. Each recording is paired with a precise reference MIDI file. The dataset covers a pitch range of five octaves (C2 to C6) and a BPM range of 52 to 156.

Comparison of HumTrans with existing humming datasets.

Compared to existing datasets like MTG-QBH (0.88 hrs) or MLEnd Hums and Whistles (29.77 hrs), HumTrans is nearly double the size of the previous largest collection. This scale is critical for training modern deep learning architectures that require vast amounts of data to generalize across different vocal timbres.

Baseline Evaluation and Key Insights

We tested four state-of-the-art (SOTA) vocal transcription models on HumTrans: VOCANO, Sheet Sage, MIR-ST500, and JDC-STP. Despite the high quality of the data, current models performed poorly. The top performer, JDC-STP, achieved a Test F1-score of only 5.66%.

This low performance highlights that humming transcription is not just a "data problem" but an algorithmic challenge. The human voice's nuances require new architectures capable of joint pitch and onset detection. We also advocate for Octave-Invariant Evaluation; since different people have different natural vocal ranges, a model that transcribes the correct melody in a different octave is still practically useful.

Conclusion

HumTrans provides the necessary "training ground" for the community to improve humming transcription. While the current data is high-quality, future work will involve expanding participant diversity to include non-musicians and a wider variety of vocalizations like whistling or "Ah" sounds. HumTrans is available on Hugging Face, with baseline code available on GitHub to encourage further research.

Summary updated

Abstract

This paper introduces the HumTrans dataset, which is publicly available and primarily designed for humming melody transcription. The dataset can also serve as a foundation for downstream tasks such as humming melody based music generation. It consists of 500 musical compositions of different genres and languages, with each composition divided into multiple segments. In total, the dataset comprises 1000 music segments. To collect this humming dataset, we employed 10 college students, all of whom are either music majors or proficient in playing at least one musical instrument. Each of them hummed every segment twice using the web recording interface provided by our designed website. The humming recordings were sampled at a frequency of 44,100 Hz. During the humming session, the main interface provides a musical score for students to reference, with the melody audio playing simultaneously to aid in capturing both melody and rhythm. The dataset encompasses approximately 56.22 hours of audio, making it the largest known humming dataset to date. The dataset will be released on Hugging Face, and we will provide a GitHub repository containing baseline results and evaluation codes.
BibTeX
@inproceedings{liu2024humtrans,
  author = {Shansong Liu and Xu Li and Dian Li and Ying Shan},
  title = {{HumTrans: A Novel Open-Source Dataset for Humming Melody Transcription and Beyond}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  doi = {10.1109/icassp48485.2024.10445818},
}
Figure