Audio Impairment Recognition Using a Correlation-Based Feature Representation

Alessandro Ragano, Emmanouil Benetos, Andrew Hines

Summary

Researchers have developed a novel method for representing audio features by mapping the correlation between pairs of features into 2D histograms. While this approach maintains classification accuracy comparable to traditional raw features, it significantly reduces feature dimensionality and accelerates inference by up to 53%. This efficiency makes it a promising technique for real-time audio monitoring and restoration.

Introduction

Identifying audio impairments—such as vinyl hiss, tape noise, or digital glitches—is essential for audio restoration and quality assessment. Traditional methods rely on "hand-crafted" features like spectral centroid or zero-crossing rate. However, these features often lack robustness; for instance, high-frequency energy from a hi-hat cymbal can be misidentified as hiss. While deep learning models can address this, they require significant data and computational resources. This paper explores a more efficient feature representation to distinguish noise from signal across diverse musical styles.

Methodology

The core innovation is a correlation-based feature representation that maps the relationships between pairs of features into 2D histograms, effectively translating audio data into a visual language suitable for Convolutional Neural Networks (CNNs).

  1. Feature Extraction: Standard short-time (ST) features (e.g., spectral centroid, zero-crossing rate) and Mel-frequency cepstral coefficients (MFCCs) are extracted.
  2. Correlation Mapping: Instead of feeding raw values into a model, pairs of features are used to generate 2D histograms over 4-second windows. This creates a "scatter plot" image representing feature relationships.
  3. CNN Input: These histograms are concatenated into a single matrix. Because this matrix represents spatial relationships between feature correlations, it is ideal for CNNs, which excel at identifying patterns in grid-like data.
  4. Resolution Tuning: The researchers evaluated histogram resolutions of 4, 8, and 16 bins to optimize the balance between detail and computational overhead.
Comparison of 2D histogram resolutions (4, 8, and 16 bins) and the corresponding feature pairs.

Experimental Setup

The study utilized two primary datasets:

  • GTZAN: A classic music informatics dataset containing 1,000 tracks across 10 genres.
  • Freesound: Used to source real-world noise samples (vinyl hiss, tape noise, gramophone noise, and white noise).

Clean tracks were mixed with these noises at various Signal-to-Noise Ratios (SNRs), ranging from 0 dB (very noisy) to 30 dB (subtle noise), to create a robust testing environment.

Key Results and Discussion

The correlation-based approach, particularly with 8 bins, yielded significant performance benefits over raw feature representations:

  • Computational Efficiency: In the test stage, the model was 53% faster when using MFCCs and 41% faster when using ST features compared to the raw baseline.
  • Reduced Data Footprint: The feature representation was approximately 30% smaller than the raw feature equivalent.
  • Comparable Accuracy: For Audio Impairment Recognition, the method achieved an accuracy of 63.57% (MFCCs), slightly outperforming the raw feature baseline of 63.05%.
  • Optimal Resolution: Using 8 bins per histogram provided the best trade-off. Fewer bins (4) lost too much information, while more bins (16) increased processing time without a significant accuracy boost.
The 8-bin histogram configuration, which provided the optimal balance between detail and computational efficiency.

Conclusion

While the correlation-based method did not significantly improve robustness across diverse musical styles as originally hypothesized, its substantial computational benefits make it highly suitable for real-time audio monitoring. Future work will investigate correlations between higher-level features (e.g., pitch or beat) and explore matrix compression techniques to further enhance processing speed.

Summary updated

Abstract

Audio impairment recognition is based on finding noise in audio files and categorising the impairment type. Recently, significant performance improvement has been obtained thanks to the usage of advanced deep learning models. However, feature robustness is still an unresolved issue and it is one of the main reasons why we need powerful deep learning architectures. In the presence of a variety of musical styles, handcrafted features are less efficient in capturing audio degradation characteristics and they are prone to failure when recognising audio impairments and could mistakenly learn musical concepts rather than impairment types. In this paper, we propose a new representation of hand-crafted features that is based on the correlation of feature pairs. We experimentally compare the proposed correlation-based feature representation with a typical raw feature representation used in machine learning and we show superior performance in terms of compact feature dimensionality and improved computational speed in the test stage whilst achieving comparable accuracy.
BibTeX
@inproceedings{ragano2020audio,
  author = {Alessandro Ragano and Emmanouil Benetos and Andrew Hines},
  title = {{Audio Impairment Recognition Using a Correlation-Based Feature Representation}},
  booktitle = {QoMEX},
  year = {2020},
  doi = {10.1109/qomex48832.2020.9123111},
}
Figure