Robust voice activity detection in stereo recording with crosstalk

Prasanta Kumar Ghosh, Andreas Tsiartas, Panayiotis G. Georgiou, Shrikanth S. NarayananInterspeech 2010 · DOI

Abstract

Voice Activity Detection (VAD) in multi-speaker environments is often compromised by crosstalk, where speech from one participant leaks into the microphone of another. This paper proposes a robust "Merge-then-Select" framework that first detects speech on a combined signal and then assigns it to the primary channel using long-term acoustic features. Experimental results on a medical interaction dataset show that this approach achieves an accuracy of 92.54%, a significant 24% absolute improvement over standard single-channel VAD methods.

Introduction

In settings such as medical consultations or business meetings, participants often use close-talking microphones. Ideally, each microphone should only capture the wearer's voice. However, physical proximity leads to crosstalk, where speech signals are recorded across multiple channels. For standard VAD algorithms designed to distinguish speech from stationary background noise, crosstalk is particularly challenging because it possesses the spectral characteristics of speech. Consequently, leaked signals are often incorrectly flagged as active speech on the wrong channel, creating "ghost" segments that degrade the performance of downstream tasks like speaker diarization and automated transcription.

Proposed Methodology: Merge-then-Select

The authors address this by decoupling the detection of speech from the identification of the speaker through a two-stage process.

1. Merged VAD

Instead of processing channels independently, the signals from the left ($x_L$) and right ($x_R$) channels are summed to create a merged signal $x_{R+L}[n]$. A standard VAD (specifically the ETSI AMR VAD option 2) is then applied to this combined signal to determine if any speech is present. This stage leverages the fact that the merged signal will contain the speech information regardless of which channel it originated from.

2. Channel Selection

Once speech is detected, the system must determine the primary source channel. The authors evaluated several proximity measures to compare the individual channels against the merged reference:

  • ENERGY: Comparing signal power; the primary speaker is expected to be louder on their dedicated microphone.
  • MFCC & IS Distance: Using Mel-Frequency Cepstral Coefficients or Itakura-Saito distance to measure the spectral similarity between the merged signal and individual channels.
  • CORR: A cross-correlation baseline.
Figure 2: Block diagram of the proposed two-stage VAD system, featuring merged signal detection followed by channel selection.

A critical component of this selection process is the use of a long-term analysis window. By aggregating distance measures over a window of $R$ frames (typically 300ms to 500ms), the system filters out short-term noise and achieves much higher stability than frame-by-frame decision-making.

Experimental Setup

The framework was evaluated using a dataset of Cross-Lingual Medical Interactions. The setup involved an English-speaking doctor and a Spanish-speaking patient communicating via an interpreter. Participants were positioned approximately 1 meter apart, leading to significant crosstalk. The corpus consisted of five sessions totaling approximately 2.5 hours of stereo audio.

Results and Discussion

The proposed method demonstrated substantial gains over traditional approaches. While a standard VAD applied to individual channels achieved only 68.73% accuracy, the Energy-based selection method with a long-term window ($R=15$) reached 92.54%.

Figure 4: VAD accuracy as a function of the long-term window size R. Performance plateaus for most features between R=15 and R=25.

Key findings include:

  • Feature Performance: Surprisingly, simple energy comparisons performed as well as or better than complex spectral measures like MFCCs.
  • Window Size: Accuracy improved significantly as the window size $R$ increased, plateauing around 15–25 frames. This confirms that temporal context is vital for distinguishing primary speech from crosstalk.
  • Reference Signal: The merged signal serves as an effective reference because its short-time spectrum is mathematically closer to the primary channel than the leaked channel.

Conclusion

The "Merge-then-Select" strategy provides a computationally efficient and highly accurate solution for VAD in crosstalk-heavy environments. By utilizing long-term temporal context, the system effectively eliminates "ghost" speech segments. Future research will focus on extending this framework to handle simultaneous overlapped speech and adapting the logic for multi-microphone arrays beyond stereo recordings.

Summary updated

BibTeX
@inproceedings{ghosh2010robust,
  author = {Prasanta Kumar Ghosh and Andreas Tsiartas and Panayiotis G. Georgiou and Shrikanth S. Narayanan},
  title = {{Robust voice activity detection in stereo recording with crosstalk}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-771},
}
Figure