Using Speech Production Knowledge for Raw Waveform Modelling Based Styrian Dialect Identification

S. Pavankumar Dubagunta, Mathew Magimai-DossInterspeech 2019

TL;DR

This paper tackles the challenging task of Styrian dialect identification using raw audio waveforms, focusing on incorporating knowledge about speech production. By leveraging techniques like source-filter decomposition and transferring articulatory features learned from English, the authors achieve performance comparable to or better than existing methods, even with limited linguistic resources.

What Problem Does This Paper Address?

Dialect identification (DID) is crucial for customizing speech technologies and has applications in forensics and user experience. However, DID is generally more difficult than language identification due to the subtle differences between dialects. Many existing approaches rely on linguistic knowledge or extensive acoustic resources, which are not always available. This paper specifically addresses the Styrian Dialect sub-challenge of INTERSPEECH 2019, which presents a scenario with limited acoustic data and no linguistic resources. The authors hypothesize that since Styrian dialects have subtle pitch accent differences, the key discriminative information lies in the variations of the vocal tract during articulation.

Methods

The paper proposes two main approaches for end-to-end raw waveform modeling for Styrian dialect identification:

  • Source-Filter Decomposition: This method aims to separate the speech signal into components related to the vocal source (e.g., vocal cord vibration) and the vocal tract (e.g., the shape of the mouth and throat).
  • Homomorphic Filtering (HFVT): This technique uses complex cepstrum analysis to remove the fast-varying voice source component, retaining the vocal tract response.
  • Linear Prediction (LP): This method models the predictable components of a speech signal, which correspond to the vocal tract structure. The difference between the original signal and the LP-estimated signal (the LP residual) contains more voice-source information. These processed signals are then fed into Convolutional Neural Networks (CNNs).

  • Transfer Learning of Articulatory Knowledge: This approach leverages articulatory features (AFs) learned from a resource-rich language (English) and transfers this knowledge to the Styrian DID task.

  • Articulatory Feature (AF) Modeling: CNNs are trained on English speech data, using phone alignments and phone-to-AF mappings, to predict articulatory properties like place of constriction, tongue height, etc.
  • Transfer Learning: The trained AF-CNNs are used in two ways for Styrian DID:
  • Model Initialization: The weights of the AF-CNNs are used to initialize the corresponding layers of CNNs for Styrian DID.
  • Feature Embeddings: The AF-CNNs are used to extract articulatory embeddings from Styrian speech, which are then used to train a linear classifier.

The paper also explores the effect of speed perturbation (SP) as a data augmentation technique, as suggested in prior work.

Datasets

  • ComParE 2019 Styrian Dialect Data Set: This dataset was used for training and evaluation. It consists of 76 minutes of training data, 34 minutes of development data, and 29 minutes of test data. Crucially, only dialect labels were provided; no transcriptions or other linguistic resources were available.
  • AMI Corpus: Used for training the articulatory feature models in English.

Key Insights and Ideas

  • Vocal Tract Dominance: The study strongly suggests that differences in vocal tract shape are more discriminative for Styrian dialect identification than voice-source characteristics.
  • Articulatory Features are Key: Transferring articulatory knowledge learned from a resource-rich language (English) significantly improves Styrian DID performance, especially when linguistic resources are scarce.
  • Raw Waveform Modeling is Viable: End-to-end modeling of raw speech waveforms, enhanced with speech production knowledge, can achieve competitive results without relying on traditional low-level descriptors (LLDs) or bag-of-audio-words (BoAW) representations.
  • Source-Filter Decomposition Benefits: Homomorphic filtering and linear prediction-based signals, when processed by CNNs, show promising results, outperforming raw speech with speed perturbation on the development set.
  • Manner of Articulation is Important: Analysis of transfer learning using AF embeddings indicates that the "manner of articulation" might carry the most distinguishing information for Styrian dialects.

Key Results

  • The proposed source-filter decomposition methods (HFVT and LPE) performed better than raw speech with speed perturbation on the development set and comparably on the test set.
  • The articulatory feature-based transfer learning approach, particularly through model initialization, generally outperformed the source-filter decomposition methods, especially on the test set.
  • The best performing method achieved an Unweighted Average Recall (UAR) of 47.3% on the development set using the "Artic" CNN architecture with LPE signals and transfer learning.
  • When using AF embeddings, the "Manner" of articulation showed the highest UAR of 45.8% on the development set with LPE signals.
  • The paper's methods were found to be comparable to or better than baseline systems using LLDs, BoAW, and sequence-to-sequence auto-encoders.

Limitations and Future Work

  • Parameter Sensitivity: The paper notes that systems can be sensitive to parameter variations, as observed in the test set scores.
  • Speed Perturbation Gains: While speed perturbation (SP) benefits raw speech, it did not show significant gains for the HFVT and LPE signals.
  • Future Work:
    • Investigate how source-filter decomposition and AF transfer learning can be applied to other tasks like natural vs. synthetic speech classification.
  • Explore these methods on languages with more abundant acoustic resources.
  • Systematically study the efficacy of modeling AFs from raw speech for tasks like speech recognition, comparing it to existing feature extraction methods.

Summary updated

BibTeX
@inproceedings{dubagunta2019using,
  author = {S. Pavankumar Dubagunta and Mathew Magimai-Doss},
  title = {{Using Speech Production Knowledge for Raw Waveform Modelling Based Styrian Dialect Identification}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure