A comparison of acoustic features for articulatory inversion

Chao Qin, Miguel Á. Carreira-PerpiñánInterspeech 2007

TL;DR

This paper investigates how to best represent speech acoustics to predict the physical movements of the vocal tract (articulatory inversion). They found that features closely related to vocal tract shape, combined with smoothing and dynamic information, significantly improve prediction accuracy, though the overall gains are modest.

What Problem Does This Paper Address?

Understanding how the vocal tract moves to produce speech sounds is crucial for many applications, from speech synthesis and coding to improving automatic speech recognition. The challenge lies in "articulatory inversion": figuring out the vocal tract's shape from the sound it produces. This is tricky because the mapping from acoustics to articulation isn't one-to-one – different vocal tract shapes can produce similar sounds. This paper tackles the fundamental question of which acoustic features and processing techniques are most effective for this inversion task, aiming to bridge the gap between the complex, often "jagged" acoustic signal and the smooth, continuous movements of the articulators.

Methods

The authors conducted a systematic empirical study comparing various acoustic feature extraction and processing techniques. Their core method involved training a Multilayer Perceptron (MLP) to map acoustic features to articulatory positions. They explored:

  • Acoustic Features: Comparing standard features like MFCC, PLP, and LPC, along with variants like LSF (Line Spectral Frequencies) and filterbank energies (FBANK).
  • Dynamic Features: Investigating whether including velocities and accelerations of acoustic features (beyond just their instantaneous values) improves inversion.
  • Window Length: Testing different durations for the short-time analysis window (e.g., 25ms to 96ms).
  • Smoothing: Applying a double filtering technique to smooth out the jaggedness of acoustic trajectories, using different cutoff frequencies.
  • Time Delay: Examining the effect of a temporal offset between acoustic and articulatory frames.

Datasets

The study utilized the MOCHA database, a multichannel articulatory speech database. This database includes synchronized acoustic (16kHz sampling rate) and articulatory data (EMA measurements of coil movements on lips, tongue, and velum at 500Hz). They used data from one female speaker with a northern English accent. The dataset was split into a training set of 10,000 frames and a testing set of 2,000 frames from unseen utterances. Silence was removed from the training set to avoid issues with the MLP learning ambiguous mappings.

Key Insights and Ideas

  • LSF and PLP Shine: Features more directly related to the vocal tract's spectral envelope, like LSF and PLP, performed best. This suggests that capturing the "shape" of the sound's spectrum is more important than highly processed, decorrelated features like MFCC for this task.
  • Smoothness is Key: The "jaggedness" of acoustic features is a major hurdle for articulatory inversion. Smoothing these trajectories significantly improves performance, as it better matches the smooth nature of physical articulator movements.
  • Dynamics Help (Mostly): Incorporating dynamic features (velocities/accelerations) generally improved accuracy, except for some specific features like PLP and RASTA-PLP.
  • Longer Windows, More Smoothing: Longer analysis windows and more aggressive smoothing led to better results, likely by reducing noise and capturing broader spectral trends.
  • A Small Time Lag Matters: A slight temporal delay (around 15ms) between the acoustic signal and the articulatory movement improved accuracy, suggesting that the acoustic wave's travel time through the vocal tract needs to be accounted for.

Key Results

The study found that the best combination of features (LSF with dynamic features, a 64-80ms window, and significant smoothing) achieved an RMSE of around 1.65 mm and a correlation of 0.71. While these are the best results reported, the improvement over other combinations was modest, with the best configuration yielding only about a 0.3 mm RMSE reduction compared to less optimal settings. The authors note that the relative ranking of features remained consistent across different experimental conditions.

Limitations and Future Work

The authors acknowledge that their study was limited to a single speaker and a specific database (MOCHA). They also used a particular smoothing method and a simple MLP for inversion. They suggest that future work could explore these aspects further, including analyzing performance across different speech sound categories and investigating more sophisticated inversion models. The consistent, albeit small, improvements across various settings indicate that while the problem is challenging, incremental gains are possible with careful feature engineering.

Summary updated

BibTeX
@inproceedings{qin2007comparison,
  author = {Chao Qin and Miguel Á. Carreira-Perpiñán},
  title = {{A comparison of acoustic features for articulatory inversion}},
  booktitle = {Interspeech},
  year = {2007},
}
Figure