Abstract
Traditional Automatic Speech Recognition (ASR) systems often struggle with the inherent variability of conversational speech, where speakers frequently deviate from standard phonetic pronunciations. This paper proposes a novel approach that models speech using parallel streams of articulatory features—such as tongue position and lip aperture—rather than monolithic phone units. By employing context-dependent decision trees within a Dynamic Bayesian Network (DBN) framework, the researchers demonstrate that feature-based models achieve significantly lower perplexity and exhibit greater robustness to limited training data compared to traditional phone-based models.
Introduction
Standard ASR systems typically rely on a fixed dictionary of pronunciations. However, real-world conversational speech is characterized by "messy" variations: sounds are often mumbled, skipped, or blended. For instance, the word "can't" may be nasalized, a sub-phonetic process that traditional models, which predict "surface phones" from "underlying phones," fail to capture accurately. The authors argue that speech variation is a gradual, sub-phonetic process best modeled at the level of individual articulatory components.
Methodology: The DBN Framework and Feature Bundling
The core of the proposed system is a Dynamic Bayesian Network (DBN) where speech is represented as multiple parallel streams of articulatory features. This architecture allows for asynchrony between different parts of the vocal tract, capturing the reality that one articulator (e.g., the velum) may move to a new target before another (e.g., the tongue) has finished its previous movement.
To manage the complexity and physical constraints of the vocal tract, the researchers introduced "feature bundling." Instead of treating all features as independent, they grouped related articulators into three bundles: Tongue (tip and body), Glottis/Velum, and Lips. This approach strikes a balance between the rigidity of phone-based models and the excessive flexibility of fully independent features.
Furthermore, context-dependent decision trees were utilized to predict feature values. These trees incorporate various contextual factors, including the previous state (inertia), the next target (anticipation), and the state of other features. Lidstone’s law was applied for smoothing to handle rare contexts encountered during testing.
Experimental Setup
The models were evaluated using data from the Switchboard Transcription Project, a subset of the Switchboard database manually labeled at a fine phonetic level. The dataset comprised approximately 90,000 frames (10ms each), representing about 15 minutes of conversational speech. Phone labels were converted into articulatory feature vectors to train the feature-based models.
Results and Discussion
The primary metric for evaluation was perplexity, where lower values indicate better predictive performance. The feature-based models consistently outperformed phone-based alternatives. Specifically, "Model 7"—which incorporates cross-feature and cross-phone context—achieved a perplexity of 1.5237, compared to 1.6449 for the best phone-based model.
A key finding was the data efficiency of the articulatory approach. As shown in the performance analysis, feature-based models maintain high accuracy even with significantly reduced training sets. This is because specific articulatory features (like "lip aperture") appear across many different phones, allowing the model to learn these components more effectively than a phone-based model can learn rare individual phones.
Conclusion
Modeling speech at the articulatory level provides a more nuanced and accurate representation of conversational pronunciation variation. By leveraging context-dependent decision trees and feature bundling, this approach achieves superior predictive power and robustness. Future work will focus on integrating these models into end-to-end speech recognition systems to evaluate their impact on Word Error Rates (WER).
Summary updated