Unsupervised Learning of Sparse Features for Scalable Audio Classification

Mikael Henaff, Kevin Jarrett, Koray Kavukcuoglu, Yann LeCunISMIR 2011 · OpenReview

Abstract

This paper introduces a scalable framework for audio feature extraction using Predictive Sparse Decomposition (PSD). By training a non-linear encoder to approximate sparse representations, the method achieves the high discriminative power of sparse coding with the computational efficiency of a feed-forward neural network. Evaluated on the GTZAN dataset, the approach yields state-of-the-art genre classification results (83.4% accuracy) and demonstrates the ability to autonomously discover fundamental musical structures from raw audio data.

1. Introduction

Traditional Music Information Retrieval (MIR) systems have long relied on hand-crafted features like Mel-Frequency Cepstral Coefficients (MFCCs). While effective, these descriptors are limited by expert assumptions and may not generalize across diverse acoustic tasks. Unsupervised feature learning offers a data-driven alternative, but existing methods like Deep Belief Networks (DBNs) and standard Sparse Coding (SC) suffer from high training complexity or slow inference times. This work addresses these bottlenecks by applying Predictive Sparse Decomposition (PSD) to audio, enabling real-time feature extraction for massive music databases.

2. Methodology

The feature extraction pipeline involves preprocessing audio into log-scaled Constant-Q Transform (CQT) spectrograms, followed by Local Contrast Normalization (LCN).

2.1 Dictionary Learning

The core of the system is an overcomplete dictionary learned from the preprocessed audio. This dictionary consists of "basis functions"—atomic sound patterns that can be linearly combined to reconstruct the input signal.

A grid of learned basis functions representing diverse atomic sound patterns discovered through unsupervised learning.

2.2 The PSD Encoder

To achieve scalability, the authors replace the iterative optimization typically required for sparse coding with a trained non-linear encoder. This encoder uses a simple feed-forward layer and a shrinkage (soft-thresholding) function to produce sparse features in a single pass. This architecture allows the system to process 8 hours of audio in less than 3 minutes, making it orders of magnitude faster than traditional SC.

3. Results and Discussion

The system was benchmarked on the GTZAN dataset, consisting of 1,000 30-second clips across 10 genres.

3.1 Classification Accuracy

The PSD-based features, aggregated over 5-second windows and classified with a linear SVM, achieved 83.4% accuracy. This result is competitive with supervised DBNs (84.3%) and significantly outperforms MFCC-based baselines. Notably, learning separate dictionaries for different octaves provided a 4% performance boost over a single global dictionary, highlighting the importance of local frequency patterns in genre discrimination.

3.2 Emergent Music Theory

A key insight of this work is the algorithm's ability to discover musical intervals without prior knowledge. The learned basis functions often exhibit peaks corresponding to minor thirds, perfect fifths, and major triads.

A learned basis function showing peaks at specific frequency intervals, illustrating the emergence of musical structures from raw data.

4. Conclusion

Predictive Sparse Decomposition offers a powerful and efficient solution for unsupervised audio feature learning. By bridging the gap between representational quality and computational speed, this method enables the deployment of sophisticated feature learning in large-scale industrial applications. Future research will focus on hierarchical architectures and temporal modeling to further enhance the capture of complex musical dynamics.

Summary updated

BibTeX
@inproceedings{henaff2011unsupervised,
  author = {Mikael Henaff and Kevin Jarrett and Koray Kavukcuoglu and Yann LeCun},
  title = {{Unsupervised Learning of Sparse Features for Scalable Audio Classification}},
  booktitle = {ISMIR},
  year = {2011},
  url = {https://openreview.net/forum?id=9unzEp75lX},
}
Figure