A Generative Product-of-Filters Model of Audio

Dawen Liang, Matthew D. Hoffman, Gautham J. MysoreICLR 2013 · OpenReview

TL;DR

This paper introduces the Product-of-Filters (PoF) model, a novel generative approach for audio spectrograms that learns a sparse decomposition of audio signals into "filters." PoF offers a data-driven alternative to traditional signal processing techniques and demonstrates strong performance in audio bandwidth expansion and speaker identification tasks.

What Problem Does This Paper Address?

Traditional audio processing often relies on decomposing complex sounds into simpler components, like an excitation signal and a linear filter (e.g., in homomorphic filtering or linear predictive coding). These decompositions are typically hand-designed using basic operations. The challenge is that these fixed decompositions might not optimally capture the intricate statistical properties of real-world audio signals, especially for tasks like reconstructing degraded audio or identifying speakers. This paper aims to bridge this gap by proposing a generative model that learns these decomposing filters directly from data, allowing for a more adaptive and potentially more accurate representation of audio.

Methods

The core of this paper is the Product-of-Filters (PoF) model. It models audio spectrograms (specifically, their log-magnitudes) as a linear combination of learned filters. This is inspired by homomorphic filtering, where a signal is a convolution of an excitation and a filter, which translates to an addition in the log-spectral domain. PoF extends this by allowing for multiple learned filters, represented by a matrix $U$.

Key methodological contributions include:

  • Generative Model Formulation: The PoF model is formally defined with probabilistic priors (Gamma distributions) on filter activations and noise levels, encouraging sparsity in filter usage.
  • Mean-Field Variational Inference: Since exact posterior inference is intractable, the authors derive a mean-field variational inference algorithm to approximate the posterior distribution of the latent filter activations. This involves minimizing the KL divergence between a tractable variational distribution and the true posterior.
  • Variational Expectation-Maximization (EM): To learn the model's free parameters (the filters $U$, sparsity parameters $\alpha$, and noise parameters $\gamma$), a variational EM algorithm is employed. This iteratively optimizes the variational parameters (E-step) and then the model parameters (M-step) to maximize a lower bound on the marginal likelihood.
  • Optimization: Both inference and parameter estimation leverage optimization techniques like L-BFGS to find local optima.

Datasets

  • TIMIT Speech Corpus: Used for training the PoF model parameters ($U, \alpha, \gamma$) and for evaluating its performance on both bandwidth expansion and speaker identification tasks. The corpus contains speech from American English speakers.
  • No new benchmarks were introduced.

Key Insights and Ideas

  • Data-Driven Filter Learning: The PoF model moves beyond hand-crafted filters by learning them directly from audio data. This allows the model to adapt to the specific statistical characteristics of the audio being analyzed, leading to potentially better representations.
  • Sparsity as a Feature: The use of sparsity-inducing priors on filter activations is crucial. It encourages the model to use only a few filters to represent each audio frame, leading to compact and interpretable representations. This aligns with the intuition that not all sound-generating or filtering mechanisms are active simultaneously.
  • Excitation vs. Filtering Filters: The learned filters often exhibit a clear distinction: some appear to model periodic excitation signals (like vocal folds), while others capture the filtering effects of the vocal tract. This provides a semantically meaningful decomposition.
  • PoF vs. NMF: The paper highlights that PoF models audio spectra as a product of filters (addition in log-domain), corresponding to convolution in the time domain. This contrasts with Non-Negative Matrix Factorization (NMF), which models spectra as an additive combination of dictionary elements (summation in time domain). This fundamental difference makes PoF better suited for modeling sequential filtering processes, while NMF is better for additive mixtures of sources.

Key Results

  • Bandwidth Expansion: PoF significantly outperforms NMF (both KL-NMF and IS-NMF) in terms of perceived audio quality (OVRL score), improving it from "poor-to-fair" to "fair-to-good" in a speaker-independent setting. While STOI (intelligibility) improvements were smaller, the OVRL gains are substantial.
  • Speaker Identification: PoF-derived features (PoFC) achieve a significant improvement in frame-level accuracy (60.5%) compared to MFCCs (49.1%). When combined with MFCCs, the accuracy further increases to 65.5%. Even with simple post-processing (median filter smoothing), PoFC alone reaches 85.0% accuracy. This suggests PoFC captures complementary information to MFCCs.

Limitations and Future Work

  • Computational Cost: The current variational EM algorithm, while parallelizable across frames, is computationally intensive, making it difficult to train on very large datasets on a single machine.
  • Local Optima: Like many dictionary learning methods, the optimization process can get stuck in local optima, although the use of multiple restarts and careful initialization helps mitigate this.
  • Future Work:
    • Stochastic Variational Inference: To handle larger datasets, the authors suggest exploring stochastic variational inference techniques.
  • Building Block: PoF can be integrated into larger models, such as serving as a prior for dictionaries in probabilistic NMF.
  • More Diverse Data: Training on larger, more diverse speech corpora or even continuous streams of data is a promising direction.
  • Sophisticated Classifiers: The success of PoFC with a simple linear SVM suggests that using it with more advanced models (e.g., deep neural networks) could yield even better results.

Summary updated

Abstract

We propose the product-of-filters (PoF) model, a generative model that decomposes audio spectra as sparse linear combinations of "filters" in the log-spectral domain. PoF makes similar assumptions to those used in the classic homomorphic filtering approach to signal processing, but replaces hand-designed decompositions built of basic signal processing operations with a learned decomposition based on statistical inference. This paper formulates the PoF model and derives a mean-field method for posterior inference and a variational EM algorithm to estimate the model's free parameters. We demonstrate PoF's potential for audio processing on a bandwidth expansion task, and show that PoF can serve as an effective unsupervised feature extractor for a speaker identification task.
BibTeX
@inproceedings{liang2013generative,
  author = {Dawen Liang and Matthew D. Hoffman and Gautham J. Mysore},
  title = {{A Generative Product-of-Filters Model of Audio}},
  booktitle = {International Conference on Learning Representations},
  year = {2013},
  url = {https://openreview.net/forum?id=4KDepjcqlOf},
}
Figure