A comparison of subspace feature-domain methods for language recognition

W. M. Campbell, Douglas E. Sturim, Pedro A. Torres-Carrasquillo, Douglas A. ReynoldsInterspeech 2008

TL;DR

This paper introduces fNAP, a novel feature-domain method for compensating for acoustic mismatch in language recognition systems. By adapting the Nuisance Attribute Projection (NAP) technique to the feature space, fNAP achieves performance comparable to existing methods like Latent Factor Analysis (LFA) while offering a new perspective on feature compensation.

What Problem Does This Paper Address?

Automatic language recognition systems often struggle with "acoustic mismatch" – variations in speech signals caused by differences in speakers, channels, gender, or recording environments. This mismatch can significantly degrade recognition accuracy. Traditional methods like RASTA and VTLN attempt to normalize features, but more recent approaches like Latent Factor Analysis (LFA) and Nuisance Attribute Projection (NAP) aim to compensate for these nuisance variations by projecting them out of a learned subspace. While LFA has been successfully adapted to the feature domain (fLFA), a similar adaptation for NAP, termed fNAP, was missing. This paper bridges that gap by proposing and evaluating fNAP, comparing its effectiveness against fLFA on a challenging language recognition task.

Methods

The core contribution is the development of fNAP (feature-domain Nuisance Attribute Projection). This method adapts the NAP technique, originally designed for SVM expansion spaces, to operate directly on acoustic features. The paper also revisits and utilizes Latent Factor Analysis (LFA), a technique that models nuisance variations in a latent subspace, and its feature-domain adaptation, fLFA.

For language recognition, the authors employ Support Vector Machines (SVMs) trained with a Gaussian Mixture Model (GMM) mean and covariance kernel. This kernel allows for the comparison of GMM supervectors, which are essentially stacked mean and covariance parameters from GMMs adapted to individual utterances.

The paper details how both fNAP and fLFA are integrated into this SVM framework. For fNAP, the NAP projection is applied in the feature domain by essentially subtracting a learned nuisance mean bias from the features. For fLFA, the latent factors are estimated and used to compensate the features. The authors also discuss a "pushed GMM" scoring method for SVMs, which involves creating separate "plus" and "minus" GMM models based on the SVM's support vectors to improve scoring robustness.

Datasets

The experiments were conducted on the NIST 2007 language recognition evaluation (LRE) data. This dataset comprises 14 languages: Arabic, Bengali, Chinese, English, Farsi, German, Hindustani, Japanese, Korean, Russian, Spanish, Tamil, Thai, and Vietnamese.

The training data primarily came from Callfriend and Callhome, with additional data from Fisher, Mixer, and NIST/LDC for specific languages. A development set (LRE07 DEV) of approximately 6000 utterances per duration (3, 10, and 30 seconds) was used for training backend fusion systems.

Key Insights and Ideas

  • Feature-domain compensation is effective: Adapting model-domain compensation techniques like NAP and LFA to the feature domain (fNAP and fLFA) yields significant improvements in language recognition accuracy, demonstrating that these nuisance variations can be effectively addressed directly in the feature space.
  • fNAP is a viable alternative: The proposed fNAP method achieves performance comparable to fLFA, offering a new and effective tool for feature compensation in language recognition.
  • Subspace methods are most impactful at scoring: The experiments reveal that applying subspace compensation methods during the scoring phase provides the largest gains in error rate reduction, suggesting that fine-tuning the decision-making process with compensated features is crucial.
  • Subspace dimension matters: There's an optimal dimension for the learned subspace. Too low a dimension may not capture enough nuisance variation, while too high a dimension can remove too much useful information, degrading performance. For this task, an optimal dimension around 64 was observed.
  • Lower mixture orders can be beneficial: Using fewer mixture components for feature extraction (e.g., 512 instead of 2048) can lead to moderately better results and significantly reduces computational cost.

Key Results

The paper demonstrates that both fNAP and fLFA significantly reduce the Equal Error Rate (EER) compared to systems without subspace compensation. For instance, full compensation (using subspace methods in training, language model, and scoring) yielded a substantial improvement of about 33% reduction in EER at 30-second test durations.

Specifically, Table 1 shows that for a 30-second duration and a subspace dimension of 64, fNAP achieved an EER of 1.39%, very close to fLFA's 1.45%. Table 3 highlights that applying subspace methods solely at the scoring stage still yields large gains. The authors also note that fNAP and fLFA generally produce similar EERs across various parameter settings.

Limitations and Future Work

  • Optimal dimension tuning: The paper identifies an optimal subspace dimension (around 64) but acknowledges that this might be dataset and task-dependent. Further research could explore more adaptive methods for determining this dimension.
  • Short duration performance: While improvements were seen for shorter utterances (10s), the gains were less pronounced than for longer ones (30s). The authors suggest this is likely due to the difficulty of estimating the nuisance space with limited test data. Future work could focus on improving compensation for short utterances.
  • Comparison to other methods: While fNAP is compared to fLFA, the paper could benefit from a broader comparison with other state-of-the-art feature compensation techniques.
  • Generalizability: The study focuses on NIST LRE data. Further validation on diverse datasets and languages would be valuable to assess the generalizability of fNAP.

Summary updated

BibTeX
@inproceedings{campbell2008comparison,
  author = {W. M. Campbell and Douglas E. Sturim and Pedro A. Torres-Carrasquillo and Douglas A. Reynolds},
  title = {{A comparison of subspace feature-domain methods for language recognition}},
  booktitle = {Interspeech},
  year = {2008},
}
Figure