Improved Model Selection for the ASR-Driven Binary Mask

William Hartmann, Eric Fosler-Lussier

Introduction

Automatic Speech Recognition (ASR) systems frequently struggle in noisy environments where background sounds overlap with speech. A common mitigation strategy involves the use of a binary mask—a digital filter applied to a spectrogram that isolates "cells" (time-frequency units) where speech is dominant while blocking those dominated by noise.

While the Ideal Binary Mask (IBM) provides a theoretical upper bound for performance, it requires access to the clean signal, which is unavailable in real-world scenarios. Most practical implementations rely on "bottom-up" signal processing (e.g., spectral subtraction) to estimate the mask. This paper explores a "top-down" alternative: leveraging phonetic hypotheses from the ASR system itself to guide mask selection, effectively using linguistic context to predict which parts of the audio contain speech.

The ASR-Driven Framework

The authors refine a multi-stage framework for mask estimation. The process begins with an initial ASR pass on the noisy audio to generate a "lattice" of possible phonetic units for every frame. Each phonetic unit is associated with a phonetic prior—a pre-learned energy distribution across 64 frequency bands. This creates a set of candidate masks for each time step.

Feature Integration and Sequence Modeling

The core innovation of this work is the transition from simple signal-matching to a discriminative sequence model. The authors introduce two major enhancements:

  1. MLP Posteriors: A Multi-Layer Perceptron (MLP) is trained to predict the probability of a time-frequency unit being masked, using both signal features and phonetic priors.
  2. Structured Perceptron: A discriminative linear chain sequence model is used to select the best sequence of masks across time. This allows the system to utilize "transition information"—the predictable patterns of speech sounds—to make more informed decisions than frame-isolated methods.
Table 1: Feature functions utilized in the structured perceptron for mask selection, including observation and transition features.

Experimental Evaluation

The proposed method was evaluated using the Aurora4 corpus, which consists of Wall Street Journal (WSJ0) recordings augmented with six different noise types (e.g., car, babble, restaurant) at signal-to-noise ratios (SNRs) ranging from 0 to 20 dB.

The Sequence Model (SM)-based approach demonstrated significant gains over the previous Spectral Subtraction (SS)-based method. As shown in the results table below, the SM-based estimate achieved an average Word Error Rate (WER) of 30.9%, representing a 14.2% relative improvement over the noisy baseline of 36.0%.

Table 2: Word Error Rate (WER) results across different noise conditions in the Aurora4 dataset, comparing various estimation methods and oracle models.

Key Insights and Future Work

The results highlight the value of linguistic context in noise-robust ASR. By tying masks to phonetic units (like "th" or "ah"), the system effectively "expects" specific energy patterns, making it more robust to non-stationary noise that might otherwise be mistaken for speech. Furthermore, sequence modeling captures the temporal dependencies inherent in speech, significantly closing the gap toward the theoretical "Oracle" performance of the ASR-driven approach (24.9% WER).

However, a performance gap still exists between the ASR-driven oracle and the Ideal Binary Mask (17.1% WER), suggesting that phonetic units alone do not capture all nuances of the clean signal. Additionally, the system's reliance on an initial ASR pass creates a "chicken and egg" problem; if the first pass is too degraded by noise, mask selection suffers. Future work may explore iterative decoding processes to refine hypotheses and masks progressively.

Summary updated

BibTeX
@inproceedings{hartmann2012improved,
  author = {William Hartmann and Eric Fosler-Lussier},
  title = {{Improved Model Selection for the ASR-Driven Binary Mask}},
  booktitle = {Interspeech},
  year = {2012},
  doi = {10.21437/interspeech.2012-372},
}
Figure