Synthetic speech discrimination using pitch pattern statistics derived from image analysis

Phillip L. De Leon, Bryan Stewart, Junichi YamagishiInterspeech 2012

TL;DR

This paper proposes a novel method to distinguish between human and synthetic speech by analyzing statistical features of the speech's pitch pattern, visualized as an image. By extracting features like mean pitch stability and jitter, they achieve high accuracy in classifying speech, offering a more robust security measure for voice authentication systems.

What Problem Does This Paper Address?

As text-to-speech (TTS) technology advances, synthetic voices are becoming increasingly indistinguishable from human voices. This poses a significant security risk, particularly for voice authentication systems, which could be vulnerable to sophisticated imposters using synthetic speech. Existing methods for detecting synthetic speech often rely on features that are difficult to generalize or require matched training data, limiting their practical application. This paper aims to address this by developing a more general and robust method for synthetic speech detection using pitch pattern analysis.

Methods

The core of this paper's contribution lies in its novel approach to feature extraction from pitch patterns. Here's a breakdown:

  • Pitch Pattern Calculation: They first compute a normalized, short-range autocorrelation function of the speech signal. This is then transformed into a binary "pitch pattern image" by applying a threshold.
  • Image Analysis for Features: Instead of directly analyzing the raw pitch pattern, the authors treat it as an image. They then perform image processing to identify and filter connected components (regions) within this binary image.
  • Statistical Feature Extraction: From these connected components, they extract three key statistical features:
  • Mean Pitch Stability ($\mu_s$): Measures the average stability of the pitch over connected components.
  • Mean Pitch Stability Range ($\mu_R$): Captures the average range of pitch variation within connected components.
  • Jitter (J): Quantifies the variance of peak lags, reflecting subtle variations in pitch timing.
  • Multivariate Gaussian Classifier: These extracted features are then used to train a classifier. Specifically, they model the distribution of synthetic speech features using a multivariate Gaussian distribution with a diagonal covariance matrix. A decision threshold is set to optimize classification accuracy.

What's new here is the application of image analysis techniques to pitch patterns and the specific set of statistical features derived from this process. This contrasts with previous work that might have used more direct pitch measurements or different feature sets.

Datasets

The paper utilizes several datasets for both training and testing:

  • Training Data:
    • Human Speech: NIST2002 corpus (330 speakers, ~30s each).
  • Synthetic Speech: * Festival pre-built voices (15 signals, 15-30s each).
  • Blizzard Challenge 2008 and 2011 (152 and 59 signals, respectively, first 30s).
  • Testing Data:
    • Human Speech: Switchboard corpus (352 speakers).
  • Synthetic Speech: * Synthetic WSJ voices (derived from HMM-based TTS, 283 speaker models).
  • Synthetic RM voices (Resource Management corpus).

The authors note that their approach is designed to be general, not requiring synthetic speech to be matched to the human speech used for training.

Key Insights and Ideas

  • Pitch Pattern as an Image: Visualizing the pitch pattern as an image and applying image processing techniques opens up new avenues for feature extraction that might capture subtle, supra-segmental characteristics of speech.
  • Statistical Features Capture Synthetic Speech Characteristics: The chosen statistical features (mean pitch stability, range, and jitter) appear to effectively differentiate between the often more rigid or hyperarticulated patterns of synthetic speech and the smoother, more natural variations in human speech.
  • Human Speech Clusters Tightly: A key observation, visualized in Figure 3, is that human speech features tend to cluster in a compact and distinct space, while synthetic speech features are more spread out but still form a discernible region. This separation is crucial for effective classification.
  • Generalizability is Key: The paper emphasizes the importance of a classifier that can generalize across different synthetic speech generation methods and corpora, avoiding the need for speaker-specific synthetic voice creation.

Key Results

The proposed method demonstrates strong performance:

  • Classification Accuracy:
    • Human Speech: 98% accuracy.
  • Synthetic Speech: 96% accuracy.
  • These results are reported to be as good as or better than previous work ([7]) while using a much larger and more diverse evaluation set. They also outperform other methods ([6]) without the need for matched training data.

Limitations and Future Work

  • Limitations:
    • The current feature vector is extracted only from voiced segments of the speech signal.
  • The classifier models the feature distributions as a Gaussian distribution.
  • Future Work:
    • Phoneme-Level Modeling: The authors suggest modeling feature vectors at the phoneme level, as they've observed significant separation in feature vectors for certain phonemes. This could lead to increased accuracy.
  • Exploring Other Features: While not explicitly stated as a limitation, the focus on pitch patterns implies that other acoustic features might also be valuable for a comprehensive detection system.

Summary updated

BibTeX
@inproceedings{leon2012synthetic,
  author = {Phillip L. De Leon and Bryan Stewart and Junichi Yamagishi},
  title = {{Synthetic speech discrimination using pitch pattern statistics derived from image analysis}},
  booktitle = {Interspeech},
  year = {2012},
}
Figure