Two-wire nuisance attribute projection

Yosef A. Solewicz, Hagai AronowitzInterspeech 2009

TL;DR

This paper tackles the challenge of accurately identifying speakers in two-wire conversations, where the presence of a second, interfering speaker significantly degrades performance. The authors introduce novel "Two-Wire Nuisance Attribute Projection" (NAP) methods that effectively remove this unwanted "speaker noise" from the data, leading to substantial improvements in recognition accuracy.

What Problem Does This Paper Address?

Speaker recognition systems are designed to identify individuals based on their voice. A common scenario is testing on recordings where a single speaker is present (four-wire conversations). However, real-world applications often involve two-wire conversations, like those in call centers or meetings, where two people are speaking simultaneously. This creates a significant problem: the voice of the other speaker acts as a major source of noise, corrupting the signal of the target speaker. Traditional speaker recognition methods, even those that handle channel variations, struggle with this "interfering speaker" noise. Simply applying standard noise reduction techniques isn't enough because the interfering speaker's voice occupies a similar "speaker space" as the target speaker, making it difficult to distinguish and remove. This paper aims to bridge this gap by developing methods specifically designed to mitigate the impact of this two-wire nuisance.

Methods

The core contribution of this paper lies in extending the Nuisance Attribute Projection (NAP) framework to handle two-wire conversations. NAP, in its original form, projects out "nuisance attributes" from speaker supervectors, typically focusing on channel variations. The authors propose two main approaches to adapt NAP for two-wire scenarios:

  1. Mathematical Variability Model: This approach starts by modeling a two-wire conversation as a linear combination of the two individual four-wire speaker supervectors. Based on this model, they derive a generalized NAP formulation that explicitly accounts for both intra-speaker (channel) variability and inter-speaker (the interfering voice) variability. The key idea is that the interfering speaker's voice creates undesirable directions in the "speaker subspace" that can be identified and removed.
  2. Explicit Modeling of Variability: Instead of a mathematical model, this method directly estimates the variability between four-wire and two-wire supervectors for the same speaker. It constructs a variability matrix that captures the differences introduced by the presence of a second speaker. The eigenvectors of this matrix then define the nuisance subspace to be projected out.

A crucial refinement in both approaches is the concept of orthogonalizing the pooled channel and speaker eigenspaces. The authors found that the channel and speaker nuisance spaces are not entirely independent. By combining the eigenvectors from both spaces and applying the Gram-Schmidt process, they create a more robust nuisance subspace, leading to what they call "Orthogonalized Eigenvoices" (OEVs).

Datasets

The experiments were conducted using subsets of the NIST Speaker Recognition Evaluation (SRE) corpora.

  • NIST SRE'04 and SRE'06 were used as development datasets for training the Universal Background Model (UBM), SVM background models, and for modeling inter-session and inter-speaker variability.
  • The male subset of the NIST-2005 SRE protocol was used for evaluation. This protocol, originally designed for four-wire sessions, was adapted for two-wire testing by artificially summing the two sides of the conversations. This adaptation introduced a unique challenge, as some four-wire impostor trials could mistakenly become two-wire target trials.

Key Insights and Ideas

  • Two-wire noise is a "speaker space" problem: The interfering speaker's voice doesn't just add random noise; it occupies directions within the speaker's own voice characteristics, making it a particularly tricky form of nuisance.
  • Projecting out "speaker space" can be beneficial: Counter-intuitively, removing parts of the speaker's own characteristic space can improve recognition in two-wire scenarios because it targets the interfering speaker's contribution.
  • Channel and speaker nuisance are correlated: The assumption of orthogonality between channel and speaker spaces, often made in traditional NAP, doesn't hold well in practice for two-wire scenarios. This correlation necessitates a method to jointly handle both.
  • Orthogonalized Eigenvoices (OEVs) are key: By combining and orthogonalizing channel and speaker eigenspaces, the authors create a more effective nuisance subspace that significantly boosts performance.

Key Results

The paper demonstrates that the proposed Two-Wire NAP methods significantly outperform regular NAP in various two-wire tasks.

  • Four-wire training vs. two-wire testing: Figure 1 shows that as the rank of OEVs (representing the projected-out nuisance) increases, the Detection Cost Function (DCF) decreases substantially for both the mathematical and explicit Two-Wire NAP approaches, outperforming standard NAP.
  • Four-wire training vs. speaker-segmented testing: Figure 2 shows that even with perfectly segmented conversations, OEV projection still improves performance, indicating its robustness.
  • Full two-wire testing: Figure 3 highlights that the explicit Two-Wire NAP approach clearly outperforms the mathematical one, and both significantly outperform standard NAP when no segmentation is used. The advantages are most pronounced in this challenging full two-wire scenario.

Limitations and Future Work

  • Assumption of linear combination: The mathematical model assumes a linear combination of supervectors, which is an approximation.
  • Orthogonality assumption: While the paper addresses the correlation between channel and speaker spaces by orthogonalizing them, the initial derivation in Section 2.1 relies on an assumption of statistical uncorrelation.
  • Future Work: The authors suggest combining the proposed Two-Wire NAP with speaker segmentation techniques. Further research into the interaction between speaker and channel spaces is also proposed to enhance the framework. They also mention that the explicit modeling approach could be used to automatically assess the segmentation quality of two-wire conversations.

Summary updated

BibTeX
@inproceedings{solewicz2009two,
  author = {Yosef A. Solewicz and Hagai Aronowitz},
  title = {{Two-wire nuisance attribute projection}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure