Modelling a Noisy-channel for Voice Conversion Using Articulatory Features

Bajibabu Bollepalli, Alan W. Black, Kishore PrahalladInterspeech 2012 · DOI

Introduction

Traditional Voice Conversion (VC) systems typically rely on parallel training data, where both source and target speakers record identical sentences. This requirement is a significant bottleneck, limiting the scalability and flexibility of VC technology. This paper proposes a "noisy-channel" framework that bypasses the need for parallel data by utilizing Articulatory Features (AFs) as a speaker-independent "canonical" representation of speech.

In this framework, speech is modeled as a message ($C$) passing through a speaker-specific "noisy channel"—the vocal tract—to produce the observed surface speech ($S$). By extracting a version of $C$ that is stripped of speaker identity, the system can be trained using only data from the target speaker to "color" any generic message with that speaker's unique vocal characteristics.

Methodology

The proposed system relies on the physical mechanics of speech, represented by AFs (e.g., manner and place of articulation). The process involves several key stages:

  1. Feature Extraction: An Artificial Neural Network (ANN) maps acoustic Mel-cepstral coefficients (MCEPs) to a 26-dimensional AF vector.
  2. Normalization via Smoothing: Raw AFs surprisingly retain significant speaker identity. To transform them into a truly canonical form, the authors apply iterative mean-smoothing (using 5-point or 11-point windows). This process removes speaker-specific "jitter" while preserving linguistic content.
  3. Speaker-Coloring Function: A target-specific ANN is trained to map these smoothed AFs back into MCEPs.
  4. Average Encoder: To maximize speaker independence, an "average speaker encoder" trained on a multi-speaker dataset is used to generate the initial AFs.
The training and conversion pipeline for the noisy-channel voice conversion model.

Key Insights: AFs and Speaker Identity

A critical finding of this research is that AFs are not inherently speaker-independent. Experiments using the TIMIT dataset revealed that a speaker identification system could achieve 85.24% accuracy using raw AFs alone. This challenged the prevailing assumption that AFs are purely linguistic.

The authors demonstrated that temporal smoothing is an effective normalization technique. As shown in the figure below, increasing the level of smoothing significantly degrades speaker identification accuracy, effectively "washing out" identity and making AFs a viable canonical form for the noisy-channel model.

Speaker identification accuracy decreases as the level of temporal smoothing on AF trajectories increases.

Experimental Results

The system was evaluated using the CMU ARCTIC dataset and cross-lingual data (Tamil and Telugu).

  • Objective Metrics: The model achieved a Mel-cepstral Distortion (MCD) between 6.3 and 8.6 dB. While this is slightly higher than parallel-data baselines (5.3–7.3 dB), it represents a strong result for a non-parallel approach.
  • Subjective Quality: Mean Opinion Scores (MOS) ranged from 3.14 to 3.40, closely trailing the parallel baseline (3.34–3.58).
  • Cross-Lingual Conversion: The system successfully converted Tamil and Telugu speech into an English target voice. Although intelligibility was lower than intra-lingual conversion, the target speaker's identity was successfully captured.

Conclusion

The noisy-channel approach offers a promising path for non-parallel voice conversion. By decoupling linguistic content from speaker identity through smoothed articulatory features, the model enables flexible conversion, including cross-lingual tasks. Future work will focus on more sophisticated normalization techniques to close the performance gap with parallel-data methods and improve intelligibility in cross-lingual scenarios.

Summary updated

BibTeX
@inproceedings{bollepalli2012modelling,
  author = {Bajibabu Bollepalli and Alan W. Black and Kishore Prahallad},
  title = {{Modelling a Noisy-channel for Voice Conversion Using Articulatory Features}},
  booktitle = {Interspeech},
  year = {2012},
  doi = {10.21437/interspeech.2012-587},
}
Figure