Improving monaural speaker identification by double-talk detection

Rahim Saeidi, Pejman Mowlaee, Tomi Kinnunen, Zheng-Hua Tan, Mads Græsbøll Christensen, Søren Holdt Jensen, Pasi FräntiInterspeech 2010 · DOI

Abstract

Identifying overlapping speakers in a single-channel recording is a complex task often hindered by the interdependence of speech separation and speaker identification. This paper presents a novel approach using a gender-dependent Double-Talk Detector (DTD) to identify high-confidence single-speaker segments within a mixture. By weighting these segments more heavily during the identification process, the system achieves a performance boost on the GRID corpus, raising accuracy from 96.53% to 97.43%. The method is speaker-independent and computationally efficient compared to exhaustive search-based alternatives.

Introduction

Identifying speakers in clean, single-source recordings is largely considered a solved problem. However, monaural speaker identification—where two voices overlap in a single channel—is considerably more difficult. Traditional approaches often attempt speech separation before identification, but effective separation typically requires prior knowledge of the speakers' identities. Conversely, state-of-the-art systems like "Iroquois" utilize exhaustive searches that are computationally expensive. This research aims to bridge this gap by introducing an efficient, speaker-independent front-end that focuses on the most reliable portions of the acoustic signal.

Methods

The proposed system architecture consists of two primary stages: Double-Talk Detection (DTD) and Speaker Identification (SID).

Figure 2: Block diagram of the proposed speaker identification system incorporating double-talk detection.

The DTD analyzes audio in short frames to test four hypotheses: silence/noise, Speaker A only, Speaker B only, or a mixture. It utilizes sinusoidal modeling and a Maximum A Posteriori (MAP) criterion. To maintain speaker independence, the DTD employs general male and female models rather than specific individual models.

The SID back-end uses MAP-adapted Gaussian Mixture Models (GMMs). It calculates Frame-Level Likelihood (FLL) and Kullback-Leibler Divergence (KLD). A key innovation is the "bonus" system: when the DTD identifies a "single-talk" segment, the system adds a bonus score to the identified speaker's total, prioritizing high-confidence data over noisy overlapping segments.

Results and Discussion

The system was evaluated using 600 two-talker mixtures from the GRID corpus across various Signal-to-Signal Ratios (SSR) from -9dB to +6dB. The researchers categorized tests into Same Gender (SG) and Different Gender (DG) scenarios.

Figure 4: Identification accuracy results comparing the system with and without Double-Talk Detection (DTD) across different scenarios.

As shown in Figure 4, the integration of DTD improved average recognition accuracy from 96.53% to 97.43%. The most significant improvements occurred in Different Gender (DG) scenarios, where the DTD effectively distinguished between male and female spectral characteristics. The results highlight the concept of selective reliability: by identifying "single-talk" regions, the system anchors its identification on the cleanest data available. Furthermore, using gender as a proxy for speaker identity allows the DTD to function without prior knowledge of the specific individuals, making it highly practical for real-world applications. The system also demonstrated robustness at low SSRs, where one speaker is significantly quieter than the other.

Conclusion

The proposed DTD-enhanced system provides a computationally efficient alternative to exhaustive search methods, achieving competitive accuracy. While the system currently struggles with same-gender speaker differentiation within the DTD stage, it serves as a powerful filter for more complex downstream tasks. Future work will focus on integrating this front-end into complete speech separation pipelines and testing against more natural, conversational datasets.

Summary updated

BibTeX
@inproceedings{saeidi2010improving,
  author = {Rahim Saeidi and Pejman Mowlaee and Tomi Kinnunen and Zheng-Hua Tan and Mads Græsbøll Christensen and Søren Holdt Jensen and Pasi Fränti},
  title = {{Improving monaural speaker identification by double-talk detection}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-140},
}
Figure