Speaker dependent mapping for low bit rate coding of throat microphone speech

Joseph M. Anand, B. Yegnanarayana, Sanjeev Gupta, M. R. KesheoreyInterspeech 2009

TL;DR

This paper tackles the challenge of making speech recorded with throat microphones (TMs) sound more natural and intelligible for low-bitrate coding. They achieve this by using a neural network to "translate" the spectral characteristics of TM speech into those of cleaner close-speaking microphone (CSM) speech, enabling more efficient and perceptually pleasing transmission.

What Problem Does This Paper Address?

In noisy environments, traditional close-speaking microphones (CSMs) struggle to capture clear speech. Throat microphones (TMs), which pick up vibrations directly from the larynx, are much more resilient to background noise and are often preferred in challenging situations like aircraft cockpits or industrial settings. However, speech captured by TMs sounds less natural and "muffled" compared to CSM speech. This is because TMs capture different aspects of the speech production process, filtering out certain high-frequency components and not fully reflecting the vocal tract's shaping of the sound. This paper aims to bridge this gap by improving the perceptual quality of TM speech for efficient, low-bitrate coding, making it more usable in scenarios where CSM is impractical.

Methods

The core of this paper's approach is a speaker-dependent mapping technique using a multilayer feed-forward neural network (MLFFNN). Here's a breakdown:

  • Feature Extraction: Speech signals from both TMs and CSMs are analyzed using linear prediction (LP) to extract LP cepstral coefficients (LPCCs). Specifically, they use a weighted version of LPCCs (wLPCCs) to reduce the emphasis on lower-order coefficients, which can dominate error calculations during neural network training.
  • Spectral Mapping: An MLFFNN is trained to learn the relationship between the wLPCCs of TM speech and the wLPCCs of corresponding CSM speech. The network is trained using pairs of TM and CSM speech features. Once trained, it can take TM speech features as input and output estimated CSM-like spectral features.
  • Coding: The estimated CSM spectral features are then converted to Line Spectral Pairs (LSPs), which are more robust to quantization errors than direct LP coefficients. These LSPs are vector quantized (VQ) using a k-means clustering algorithm to create a codebook. The output of the VQ process is a sequence of codebook indices.
  • Transmission: For efficient transmission, only the codebook indices, along with the pitch and energy contours derived from the original TM signal, are sent. This allows for a low bit-rate representation.
  • Synthesis: At the receiver, the codebook indices are used to retrieve the corresponding LSP vectors from the codebook. These LSPs are converted back to LP coefficients, representing the estimated CSM spectra. A synthetic residual signal is generated using the pitch and energy contours from the TM signal, and this residual excites the all-pole model derived from the estimated LPCs to reconstruct the speech.

Datasets

  • Training Data: 5 minutes of speech data collected simultaneously from a single speaker using both TM and CSM.
  • Testing Data: Another 5 minutes of speech data from the same speaker.

The paper notes that TM speech data could be collected in environments with higher background noise for testing, as it's less affected by noise. The crucial aspect for training is the alignment of TM and CSM speech signals.

Key Insights and Ideas

  • Neural Networks as "Translators": The paper demonstrates that MLFFNNs can effectively learn and mimic the complex spectral transformations needed to convert the "muffled" TM speech into a more natural, CSM-like representation. This is a powerful application of neural networks for speech signal enhancement.
  • LSPs for Robust Quantization: Using Line Spectral Pairs (LSPs) for vector quantization is a key insight. LSPs are known to be more tolerant to quantization errors than direct LP coefficients, leading to more stable synthesis filters and better preservation of spectral characteristics even at low bitrates.
  • Preserving Prosody: By transmitting the pitch and energy contours from the original TM signal, the system ensures that the synthesized speech retains the natural prosody (rhythm, stress, and intonation) of the original TM recording, even though the spectral characteristics are modified.
  • Speaker-Dependent Approach: The reliance on a speaker-dependent MLFFNN means the system is optimized for a specific speaker. While this improves performance for that speaker, it implies a need for retraining for new speakers.

Key Results

  • Improved Spectral Similarity: Objective measures (Itakura distance) show a significant reduction in the spectral distance between the reconstructed speech and the original CSM speech when using the proposed mapping and coding method, compared to using TM speech directly. The distance between CSM and mapped spectra was roughly halved compared to CSM and TM spectra.
  • Low Bit-Rate Achieved: The proposed method can achieve bit-rates of approximately 1.5 kbps, with codebook indices contributing around 1 kbps and pitch/energy contours making up the rest.
  • Perceptible Quality: The synthesized speech, while not perfect, was observed to be of "perceptible quality" and intelligible.
  • Quantization Impact: The study found that vector quantization of LSPs did not significantly degrade the spectral characteristics of the mapped speech.

Limitations and Future Work

  • Residual Signal Synthesis: The authors acknowledge that the primary limitation lies in the synthesis of the residual signal (the source component of speech). Using a single residual template and modifying it can lead to discontinuities at concatenation points, affecting perceptual quality.
  • Future Work: The paper suggests focusing on alternative techniques for generating a more perceptually pleasing residual signal to improve the overall quality of the synthesized speech. Developing methods for more robust residual synthesis is a key area for future research.
  • Speaker Dependency: The current approach is speaker-dependent. Future work could explore speaker-independent mapping techniques or methods for rapid adaptation to new speakers.

Summary updated

BibTeX
@inproceedings{anand2009speaker,
  author = {Joseph M. Anand and B. Yegnanarayana and Sanjeev Gupta and M. R. Kesheorey},
  title = {{Speaker dependent mapping for low bit rate coding of throat microphone speech}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure