Spatial separation of speech signals using amplitude estimation based on interaural comparisons of zero-crossings

Hyung‐Min Park, Richard M. SternSpeech 2008 · DOI

TL;DR

This paper introduces a novel algorithm, Zero-Crossing-based Amplitude Estimation (ZCAE), for separating speech signals in noisy environments. By leveraging interaural comparisons of zero-crossings to estimate interaural time differences (ITDs), ZCAE achieves more robust signal separation and improved speech recognition accuracy compared to traditional cross-correlation methods, especially when using continuous weighting functions.

What Problem Does This Paper Address?

Automatic Speech Recognition (ASR) systems perform well in clean acoustic environments but degrade significantly in realistic scenarios with background noise, interfering speech, or other distortions. While many noise-robustness techniques exist, they often struggle with dynamically changing environments and the presence of competing speech sources. A key challenge is effectively separating the desired speech signal from interfering sounds, a task humans perform remarkably well using auditory scene analysis (ASA). Traditional computational ASA approaches often rely on interaural time differences (ITDs) derived from cross-correlation, but this paper proposes an alternative method using zero-crossings, which is hypothesized to be more robust, especially in noisy conditions. The goal is to improve ASR performance by enhancing the separation of the target speech signal from interfering sources.

Methods

The core of the proposed method is the Zero-Crossing-based Amplitude Estimation (ZCAE) algorithm. Here's a breakdown of its key components:

  • Frequency Analysis: Input audio signals from two sensors are first passed through a bank of Gammatone filters, simulating human cochlear filtering. This breaks down the signal into different frequency bands.
  • Zero-Crossing Detection: Within each frequency band, zero-crossings (points where the signal crosses the zero amplitude axis) are detected.
  • ITD Estimation from Zero-Crossings: The interaural time difference (ITD) for each frequency channel is estimated by calculating the time difference between zero-crossings in the left and right sensor signals. This is a novel approach compared to standard cross-correlation methods.
  • Analytical Relationship between ITD and Signal Strength: The paper derives an analytical, monotonic function that maps the estimated ITD in each time-frequency segment to the relative contribution (or signal strength) of the desired target source. This is a key innovation, moving beyond empirical derivations.
  • Continuously-Variable Weighting: Instead of using binary masks (which either include or exclude a time-frequency segment entirely), ZCAE employs continuously-variable weighting factors. These weights are proportional to the estimated power of the target signal, allowing for smoother transitions and more nuanced signal reconstruction.
  • Signal Reconstruction: The enhanced signal is reconstructed by combining the weighted time-frequency segments. The process involves time-reversing, convolving with Gammatone filters, and then summing across frequencies.

The paper also compares this approach to:

  • Cross-correlation-based ITD estimation: The traditional method for estimating ITDs.
  • Binary masks: A simpler approach to signal separation that assigns a binary value (0 or 1) to each time-frequency segment.
  • Missing-feature reconstruction: A technique used in conjunction with binary masks to "fill in" unreliable spectral components.

Datasets

  • DARPA Resource Management (RM1) database: Used for training and testing the ASR system.
  • CMU SPHINX-III speech recognition system: The ASR system employed in the experiments.

The paper does not introduce new benchmarks but rather evaluates its proposed method on a standard ASR task with simulated noise and interference.

Key Insights and Ideas

  • Zero-crossings as a robust ITD cue: The paper strongly suggests that zero-crossing information provides more reliable ITD estimates than traditional cross-correlation, especially in noisy or low-signal-to-noise ratio (SNR) conditions. This is a significant departure from established methods.
  • Analytical derivation for signal strength: Developing an analytical, rather than empirical, relationship between ITD and signal strength is a crucial theoretical contribution. This allows for a more generalizable and potentially more accurate weighting function.
  • Continuous weighting outperforms binary masking: The results consistently show that using continuously-variable weighting factors leads to significantly better speech recognition accuracy than binary masks. This highlights the importance of smooth transitions and nuanced signal reconstruction.
  • The synergy of ZCAE and continuous weighting: The best performance is achieved when combining zero-crossing-based ITD estimation with continuously-variable weighting, demonstrating the combined benefits of these novel approaches.

Key Results

  • Reduced ITD variability: The sample standard deviation of ITD estimates using the zero-crossing method was significantly lower than that obtained using cross-correlation, particularly in challenging signal strength conditions (Fig. 3).
  • Improved ASR accuracy: Experiments showed that the ZCAE method, using continuously-variable weighting and zero-crossing-based ITDs, achieved the lowest Word Error Rates (WERs) across various conditions, including different signal-to-interference ratios (SIRs) and the presence of background noise (Figs. 4, 5, 6).
  • Superiority over baseline and binary masks: The ZCAE approach consistently outperformed both the no-processing baseline and methods using binary masks, especially at lower SIRs.
  • Missing-feature reconstruction benefits: While binary masks with missing-feature reconstruction showed improvement over plain binary masks, they still lagged behind the continuous weighting approach.

Limitations and Future Work

  • Single interfering source: The current ZCAE approach is limited to scenarios with a single interfering source.
  • Absence of reverberation: The method has not yet been evaluated in environments with reverberation.
  • Future Work: The authors express encouragement from their results and are actively working to extend the ZCAE method to handle multiple interfering sources and reverberant conditions. They also aim to further explore the analytical derivation and its applications.

Summary updated

BibTeX
@inproceedings{park2008spatial,
  author = {Hyung‐Min Park and Richard M. Stern},
  title = {{Spatial separation of speech signals using amplitude estimation based on interaural comparisons of zero-crossings}},
  booktitle = {Speech},
  year = {2008},
  doi = {10.1016/j.specom.2008.05.012},
}
Figure