Speech style and speaker recognition: a case study

Marco Grimaldi, Fred CumminsInterspeech 2009

TL;DR

This paper investigates how different speech styles, including intentionally altered voices, impact the performance of speaker recognition systems. They found that while systems are robust to style variations when the communication channel is consistent, performance degrades significantly when both style and channel are mismatched, highlighting a critical challenge for forensic applications. The study also introduces and evaluates a novel feature extraction method, pyknogram frequency estimate coefficients (pykfec), which shows promise in improving robustness to these variations.

What Problem Does This Paper Address?

Speaker recognition systems are designed to identify or verify individuals based on their voice. These systems are typically evaluated under controlled conditions, assuming consistent speaking styles and communication channels. However, real-world scenarios, particularly in forensic contexts, are far less predictable. Speakers might be under stress, intentionally disguise their voices, or speak in different styles (e.g., fast speech, whispering). Furthermore, the communication channel itself (e.g., microphone quality, background noise) can introduce distortions. Existing research has explored channel variations extensively, but the impact of intentional speech style variations, especially when combined with channel mismatches, remains less understood. This paper aims to systematically assess how different speech styles, including imitative and fast speech, affect both speaker identification and verification systems, and to evaluate a new feature extraction method designed to be more robust to such variations.

Methods

The researchers employed a standard speaker recognition framework using Gaussian Mixture Models (GMMs) for both speaker identification (closed-set) and speaker verification (open-set).

  • Speech Parameterization: They compared two feature extraction methods:
  • MFCCs (Mel-Frequency Cepstral Coefficients): A widely used standard in speech processing.
  • pykfec (Pyknogram Frequency Estimate Coefficients): A novel parameterization proposed in prior work, derived from an AM/FM approximation of the speech signal. It's claimed to be robust to channel variations and doesn't require explicit channel compensation.
  • Speaker Identification: A generic identification system was trained and tested under various conditions:
  • Matched style and channel.
  • Mismatched style but matched channel.
  • Mismatched style and channel.
  • Speaker Verification: A similar evaluation was conducted for a verification system, using a Universal Background Model (UBM) and claimant models. The performance was assessed across matched and mismatched style/channel conditions.
  • Delta Features: The study also investigated the impact of adding delta (time derivative) features to both MFCCs and pykfec.

Datasets

The primary dataset used is the CHAINS corpus.

  • CHAINS Corpus: This corpus contains recordings of 36 speakers, collected over two sessions separated by 2-3 months. For each speaker, recordings were made in six different speaking styles.
  • Styles Used: * NORM (Normal Speech): Subjects read prepared text without specific constraints.
  • SYNC (Synchronous Speech): Two subjects read text in synchrony, typically at a slower rate.
  • RSI (Repetitive Synchronous Imitation): Subjects repeat a short phrase they hear, aiming to match it closely.
  • FAST (Fast Speech): Subjects speak at a rapid rate.
  • Training and Test Sets: The paper describes specific splits of the CHAINS corpus for training and testing, varying by session, style, and speaker roles (claimant, UBM, impostor for verification).

Key Insights and Ideas

  • Style vs. Channel Impact: Speaker recognition systems are surprisingly robust to variations in speech style as long as the communication channel remains consistent. The real performance killer is when both style and channel are mismatched.
  • Forensic Relevance: The findings have significant implications for forensic speaker recognition, where uncontrolled recording conditions and potential voice disguise are common.
  • pykfec Promise: The novel pykfec feature extraction method shows a consistent positive effect on system accuracy, particularly when dealing with mismatched conditions, suggesting it's a promising avenue for more robust speaker recognition.
  • Delta Features Help: Adding delta features generally improves performance, especially in challenging mismatched scenarios, indicating that capturing temporal dynamics is crucial.

Key Results

  • Matched Conditions: When training and testing with speech matched in both style and channel (from the same recording session), both MFCCs and pykfec achieved near-perfect accuracy (around 100%) in identification tasks.
  • Mismatched Style (Matched Channel): A decrease in accuracy was observed when styles were mismatched but the channel was the same. For example, in identification, accuracy dropped but remained relatively high.
  • Mismatched Style and Channel: This scenario led to significant performance degradation. For instance, in identification, using NORM speech for training and RSI for testing with MFCCs resulted in scores around (58±3)%, while pykfec improved this to (74±2)%.
  • Verification Performance: Similar trends were observed in speaker verification, with pykfec generally outperforming MFCCs, especially in mismatched conditions. For example, in one scenario (Train/UBM:FAST, Test/Impostor:RSI), pykfec achieved an accuracy of 84±2% compared to MFCCs' 65±2%.
  • pykfec Advantage: Across various mismatched scenarios, pykfec consistently showed better or comparable performance to MFCCs, often with a notable gain in prediction scores, particularly when delta features were included.

Limitations and Future Work

  • Limited Styles: While four styles were tested, the CHAINS corpus offers six. Further evaluation with the full set of styles could provide a more comprehensive understanding.
  • Specific Corpus: The study is based on the CHAINS corpus. Generalizing these findings to other corpora and a wider range of speakers and styles would be beneficial.
  • Forensic Realism: While the study simulates mismatched conditions, truly uncontrolled forensic scenarios might involve more complex and unpredictable variations than those tested.
  • Future Work: The authors suggest further exploration of pykfec and its potential for channel compensation. Investigating the impact of more extreme voice disguises and combining different robustness techniques are also promising directions.

Summary updated

BibTeX
@inproceedings{grimaldi2009speech,
  author = {Marco Grimaldi and Fred Cummins},
  title = {{Speech style and speaker recognition: a case study}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure