A Computationally Efficient and Practically Feasible Two Microphones\n Blind Speech Separation Method

Chandan K. Reddy, Gautam Bhat, Nikhil Shankar, Issa PanahiarXiv 2018 · DOI · arXiv

Abstract

Traditionally, Blind Speech Separation techniques are computationally expensive as they update the demixing matrix at every time frame index, making them impractical to use in many Real-Time applications. In this paper, a robust data-driven two-microphone sound source localization method is used as a criterion to reduce the computational complexity of the Independent Vector Analysis (IVA) Blind Speech Separation (BSS) method. IVA is used to separate convolutedly mixed speech and noise sources. The practical feasibility of the proposed method is proved by implementing it on a smartphone device to separate speech and noise in Real-World scenarios for Hearing-Aid applications. The experimental results with objective and subjective tests reveal the practical usability of the developed method in many real-world applications.

Introduction

In noisy environments—like a crowded restaurant—hearing aid users struggle to isolate a single voice. Blind Speech Separation (BSS) is a signal processing technique designed to solve this by "un-mixing" audio signals from multiple microphones. A popular method called Independent Vector Analysis (IVA) is highly effective at this, but it is computationally "heavy."

Traditionally, IVA updates its "demixing matrix" (the mathematical formula that separates the sounds) for every single 20ms frame of audio. On a smartphone, this constant recalculation creates an "algorithmic delay" longer than the audio frame itself, making real-time playback impossible. The authors identified that this is largely a waste of energy—acoustic conditions don't usually change every 20 milliseconds.

Proposed Methodology

The authors propose a "triggered" update system. Instead of blindly recalculating the demixing matrix, the system only does so when it detects a significant change in the speaker's position.

  1. FNNDOA (Feed Forward Neural Network for Direction of Arrival): The researchers built a tiny, efficient neural network (just one hidden layer with 8 nodes). It takes the cross-correlation of the two microphone signals as input and predicts which of seven angular zones (0° to 180° in 30° increments) the speech is coming from.
  2. The Update Criterion: The system monitors the DOA. If the speaker moves by at least 30°, the system triggers an IVA update. To prevent "jitter" or false positives from noise, they use a "4 out of 5" rule: the new angle must be detected in at least 4 out of the last 5 frames before the math is recalculated.
  3. IVA Integration: When the speaker is stationary, the system reuses the previous demixing matrix, which requires almost zero extra CPU power. When they move, the system performs the iterative IVA update to adapt to the new position.

Experimental Setup

To train and test the system, the authors used:

  • Clean Speech: Sourced from the well-known TIMIT and HINT databases.
  • Simulated Environments: They used the Image Source Method (ISM) toolbox to simulate various room acoustics, placing virtual speakers at different angles (0°, 45°, 90°, 135°, and 179°).
  • Noise Types: They tested against "Babble" (crowd noise) and "Machinery" noise at various Signal-to-Noise Ratios (SNR) from -5 dB to 5 dB.

Results and Discussion

The core innovation lies in exploiting temporal redundancy: the impulse response between a speaker and a microphone is relatively static. By choosing 30-degree increments for the DOA, the authors created a "buffer" that makes the system robust without wasting resources on micro-movements.

  • Massive Efficiency Gains: In a scenario where a speaker moves every 3 seconds, the proposed method is 37 times more computationally efficient than traditional online IVA.
  • High Audio Quality: Objective metrics like PESQ (Perceptual Evaluation of Speech Quality) and SDR (Signal to Distortion Ratio) showed the method performed nearly as well as "batch" IVA (which processes the whole file at once and can't run in real-time).
  • User Approval: In subjective Mean Opinion Score (MOS) tests with 10 human subjects, the proposed method consistently outperformed standard noisy speech and other common enhancement techniques, particularly in machinery noise.

Conclusion

The entire pipeline was designed to fit within the 20ms latency window required by Android's audio framework, proving that sophisticated BSS is feasible on mobile hardware. Future work could explore finer angular resolutions or environments with single point-source noise. The "triggered update" logic could also be applied to other complex variations of IVA to enhance their mobile-friendliness.

Summary updated

Abstract

Traditionally, Blind Speech Separation techniques are computationally\nexpensive as they update the demixing matrix at every time frame index, making\nthem impractical to use in many Real-Time applications. In this paper, a robust\ndata-driven two-microphone sound source localization method is used as a\ncriterion to reduce the computational complexity of the Independent Vector\nAnalysis (IVA) Blind Speech Separation (BSS) method. IVA is used to separate\nconvolutedly mixed speech and noise sources. The practical feasibility of the\nproposed method is proved by implementing it on a smartphone device to separate\nspeech and noise in Real-World scenarios for Hearing-Aid applications. The\nexperimental results with objective and subjective tests reveal the practical\nusability of the developed method in many real-world applications.\n
BibTeX
@misc{reddy2018computationally,
  author = {Chandan K. Reddy and Gautam Bhat and Nikhil Shankar and Issa Panahi},
  title = {{A Computationally Efficient and Practically Feasible Two Microphones\textbackslash\{\}n Blind Speech Separation Method}},
  year = {2018},
  eprint = {1812.03914},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1812.03914},
}
Figure