Abstract
Traditional speech recognition models often struggle with the inherent variability of human speech, where sounds overlap and timing shifts based on speaking rate. This paper presents a novel approach using Finite State Machines (FSMs) to model pronunciation as a sequence of overlapping articulatory gestures. By encoding timing variability into the FSM structure, the proposed system achieves over 90% word classification accuracy on synthesized speech data, demonstrating significant improvements over rigid sequence models and local bigram approaches.
Introduction
Automatic Speech Recognition (ASR) systems typically represent speech as a linear sequence of discrete phonetic units. However, this representation fails to capture the physical reality of speech production. Human speech is composed of "gestures"—coordinated movements of vocal tract articulators like the lips, tongue, and glottis. While the set of gestures required for a specific word remains relatively constant, their relative timing is highly variable due to factors like coarticulation and speech rate.
Articulatory Phonology suggests that these gestures are the fundamental units of speech. The challenge lies in creating a computational model that accommodates this "stretchy" timing while maintaining the phonological identity of the word. This paper addresses this by using Weighted Finite State Machines to represent the space of plausible gestural timings.
Methodology
The proposed system transitions from acoustic observations to articulatory representations using a multi-stage pipeline.
Gestural Pattern Vectors (GPVs)
Instead of phones, the system utilizes Gestural Pattern Vectors (GPVs). A GPV represents a snapshot of active articulatory gestures across eight tract variables (e.g., lip aperture, glottis state). The system employs ANN-GMM tandem models to estimate the likelihood of these GPVs from continuous speech observations.
FSM-Based Pronunciation Modeling
The core of the methodology is the FSM-based pronunciation model. Starting from a "Canonical Gestural Score" (CGS)—the ideal timing for a word—the system recursively generates alternative timing schemes for gesture onsets and offsets.
As shown in Figure 2, each path through the FSM represents a valid pronunciation variant. To manage complexity, the FSM is pruned using an empirical likelihood measure, ensuring that only physiologically plausible timing deviations are retained. Self-loops are included in the states to account for varying durations of individual gestures. Recognition is then performed by finding the best match between the recovered gestural activations and the word-specific FSMs.
Experimental Setup
The model was evaluated using a speech dataset synthesized by the TADA (Task Dynamic) model, based on the Wisconsin X-ray microbeam database. The dataset comprised 416 words, with 277 used for training observation models and 139 for testing classification. This synthetic data provides ground-truth articulatory movements, allowing for precise evaluation of the model's ability to recover hidden physical parameters.
Results and Discussion
The FSM-based approach significantly outperformed baseline models in word classification tasks:
- FSM-based Model (High Flexibility): 90.65% accuracy.
- Canonical GPV Sequence (Rigid Timing): 87.77% accuracy.
- GPV Bigram Model (Local Transitions): 84.89% accuracy.
The results indicate that capturing long-term gestural constraints via FSMs is superior to local bigram modeling. Furthermore, the FSM model demonstrated high accuracy in "CGS recovery," correctly identifying underlying physical targets across nearly all tract variables.
Conclusion and Future Work
This research demonstrates that FSMs can effectively bridge the gap between articulatory theory and computational speech recognition. By modeling the inherent timing variability of speech gestures, the system achieves high accuracy and robust recovery of articulatory parameters. Future work will focus on extending this model to natural human speech, addressing the challenges of inter-speaker anatomical variability and environmental noise.
Summary updated