Data Cleansing with Contrastive Learning for Vocal Note Event Annotations

Gabriel Meseguer-Brocal, Rachel M. Bittner, Simon Durand, Brian BrostarXiv 2020 · arXiv

Introduction: The Challenge of Noisy Music Annotations

In Machine Learning, the quality of training data is paramount—a principle often summarized as "garbage in, garbage out." In the field of Music Information Retrieval (MIR), obtaining high-quality "ground truth" labels is exceptionally difficult. Large-scale datasets like DALI or LakhMIDI frequently rely on crowd-sourced MIDI files or automatic alignments, which are often riddled with errors such as shifted timings, incorrect pitches, or missing notes.

Standard data cleansing techniques typically treat data points as independent samples. While effective for tasks like image classification, this approach fails for music because musical labels are inherently structured. A note at one time step is highly correlated with the note at the next. Errors in music are rarely random; they often manifest as "locally stable" patterns, such as a singer starting a fraction of a second late. This research addresses the lack of cleansing tools that account for this temporal and structural context.

Methodology: A Contrastive Learning Approach

The researchers frame data cleansing as a contrastive learning problem. They developed a binary classifier, $g(x, \hat{y})$, designed to predict the probability that a specific audio-annotation pair is incorrect.

Overview of the data cleansing pipeline, showing the generation of artificial deformations and the training of the error detection model.

Input Representation and Training

The model utilizes the Constant-Q Transform (CQT)—a spectral representation of audio—of both the full musical mixture and isolated vocals. This is paired with a binary matrix representing the annotated notes. To train the model without a pre-existing "perfect" dataset, the authors employed a two-step proxy strategy:

  1. Positive Examples ("Likely Correct"): A high-performing, pre-trained pitch estimator acts as a proxy. If the existing labels and the proxy's predictions align closely, the data is labeled as "likely correct."
  2. Negative Examples ("Incorrect"): The researchers take the "likely correct" samples and apply realistic artificial deformations, such as shifting start/end times, altering frequencies, or randomly adding and deleting notes.

Model Architecture and Application

The core of the system is a Convolutional Neural Network (CNN) that processes audio-label pairs using a large temporal context (approximately 0.94 seconds) to capture the underlying structure of the notes. Once trained, the model can be used to either filter out suspicious data (discarding frames with an error probability > 0.5) or weight the data, assigning less importance to suspicious frames during the training of downstream models.

Experimental Results

The effectiveness of the cleansing model was evaluated by training a standard vocal pitch estimator (Deep Salience) on the DALI dataset (v2), which contains over 7,700 tracks with user-submitted annotations. Performance was then tested on the MedleyDB and iKala evaluation sets.

Comparison of transcription accuracy across different data cleansing strategies, showing significant improvements when using weighted or filtered data.

Key Findings

  • Transcription Boost: Training on the "cleansed" version of the DALI dataset improved the Raw Pitch Accuracy (RPA) by over 10 percentage points compared to training on the raw, noisy data.
  • Cleanser Accuracy: The error detection model achieved a frame-level accuracy of 72.1% on a holdout set of "likely correct" vs. "artificially deformed" pairs.
  • Quantifying Dataset Noise: The model allowed for an unprecedented analysis of the DALI dataset, estimating that roughly 21.3% of its frames are likely misannotated. Furthermore, it identified that 31.1% of tracks in DALI contain more than 25% errors.

Discussion and Conclusion

The core insight of this work is that a model does not need to know the correct answer to recognize a wrong one. By learning the relationship between audio textures and label shapes, the model identifies when a label "looks" out of place—for instance, a pitch label appearing where there is no vocal energy. The results suggest that weighting samples is often more effective than simple filtering, as it provides a more nuanced signal for training.

While currently focused on vocal note events and partially dependent on source separation, the logic of this contrastive cleansing method could potentially extend to other structured musical data like chords or beats. This tool offers a path toward streamlining manual labeling by highlighting "suspicious" regions for expert review, significantly reducing the cost and effort required to create high-quality MIR datasets.

Summary updated

Abstract

Data cleansing is a well studied strategy for cleaning erroneous labels in datasets, which has not yet been widely adopted in Music Information Retrieval. Previously proposed data cleansing models do not consider structured (e.g. time varying) labels, such as those common to music data. We propose a novel data cleansing model for time-varying, structured labels which exploits the local structure of the labels, and demonstrate its usefulness for vocal note event annotations in music. %Our model is trained in a contrastive learning manner by automatically creating local deformations of likely correct labels. Our model is trained in a contrastive learning manner by automatically contrasting likely correct labels pairs against local deformations of them. We demonstrate that the accuracy of a transcription model improves greatly when trained using our proposed strategy compared with the accuracy when trained using the original dataset. Additionally we use our model to estimate the annotation error rates in the DALI dataset, and highlight other potential uses for this type of model.
BibTeX
@misc{meseguerbrocal2020data,
  author = {Gabriel Meseguer-Brocal and Rachel M. Bittner and Simon Durand and Brian Brost},
  title = {{Data Cleansing with Contrastive Learning for Vocal Note Event Annotations}},
  year = {2020},
  eprint = {2008.02069},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2008.02069},
}
Figure